A few years into the container boom, most infrastructure teams hit the same wall: they weren't struggling to run containers; they were struggling to run all of them at once. The fix wasn't a better container. It was Kubernetes, a layer whose entire job was to sit above the sprawl and make it governable.

I think enterprise telemetry is having its Kubernetes moment right now, except almost nobody has built the equivalent layer yet.

Here's the pattern I keep running into. A company's data volumes climb for years inside the comfort zone of whatever platform they bought — a SIEM, an observability suite, doesn't matter which. Then AI shows up. Not as one project, but as five: an AIOps rollout, a security analytics upgrade, a compliance monitoring mandate, a model evaluation initiative, each spinning up its own telemetry stream on top of everything already running. The platform that comfortably absorbed years of steady growth suddenly can't keep pace with a few quarters of AI-driven expansion.

The reflex, every time, is to buy a bigger platform. I watched this play out recently with a customer running security operations at real scale. Telemetry had been climbing steadily, safely inside what their SIEM could handle, until it reached the point where the incumbent system was struggling. The conversation internally was to increase capacity. That would have been the wrong fix. What they truly needed was something in front of the SIEM, not more SIEM. Look closely at what's flowing through most environments, and you'll find most of it is noise by volume, signal by exception. Web server logs are a good example, where the overwhelming share of entries are routine 2xx confirmations that nobody will ever act on. Condense that at the source, before it reaches the platform, and you've solved most of the volume problem without touching detection quality. Do the same across Windows events, application logs, and infrastructure data, and total volume drops by 40 percent or more. The order of operations matters enormously here: filtering has to happen upstream of the platform, not as cleanup after the fact.

That's the tactical fix. The bigger issue is structural, and it's about where you've allowed vendor lock-in to live.

For most of the last decade, the collection layer just came bundled with whatever platform you bought. Nobody thought about it much, because switching platforms wasn't something companies did often. AI has broken that assumption. New tools, new vendors, and new analytical capabilities are entering the market fast enough that platform evaluation is now a constant activity, not a once-a-decade event. If your data collection is welded to your current platform, every one of those evaluations turns into a six-month re-instrumentation project before you've learned anything useful. Decouple the two — separate how you collect and route data from where it ultimately lands — and evaluating a new platform becomes a configuration change instead of a migration. That's the actual strategic asset: optionality, on a timeline of days rather than months.

OpenTelemetry has quietly solved the instrumentation half of this problem. It's become the standard everyone is converging on, and it's genuinely good news. But OTel is a format, not a control plane. The organizations getting real value are the ones building a pipeline on top of it: something that gives real-time visibility into data while it's moving, lets you redirect that data without re-instrumenting the systems producing it, and holds up operationally at production scale.

None of this is free. Standardizing a schema and getting every application and infrastructure team to actually adopt it is genuinely hard, and I don't want to undersell that. But the payoff isn't incremental — it’s a different cost structure entirely that reshapes organizational operations. Compliance shifts from something teams argue about to something tooling enforces. Engineers stop spending their time on ingestion plumbing and start working on problems that actually require their judgment.

There's a version of this that matters even more once you add AI compliance into the picture. AI workloads generate their own telemetry categories — prompts, responses, model evaluation data — and regulators, especially in the EU, are getting increasingly specific about where that data can and can't go. A unified pipeline enforces residency rules from one place, across every category of data. A fragmented one means retrofitting the same rule across a dozen incompatible systems, one at a time, forever.

Regulations don't care how tidy your architecture is. Somebody has to enforce the rule. The question every organization is answering right now whether they realize it or not, is whether that somebody is going to be them, or twelve different vendors who were never coordinating in the first place.

‍