Why Engineers Still Don't Know What Metrics Their Systems Emit

The hidden chaos behind modern metrics

Observability promises clarity. Dashboards, alerts, and traces should give engineers a clear picture of what's happening inside their systems. But ask a surprisingly simple question, what metrics does our stack actually emit?, and things quickly fall apart.

One engineer described this problem well while building a tool to solve it. Modern observability stacks generate thousands of metrics across different layers: application instrumentation, exporters, Kubernetes components, and cloud platforms. Yet those metrics are scattered across hundreds of repositories, documentation pages, and instrumentation libraries.

That leaves a strange paradox: systems produce enormous amounts of telemetry, yet engineers often struggle to understand what signals even exist. It's more than a small inconvenience, because it affects how teams design monitoring strategies, build alerts, and debug incidents. It also exposes a flaw in how observability ecosystems evolved.

The metric sprawl nobody planned for

Modern infrastructure stacks are assembled like digital Lego sets. A typical environment might include Kubernetes clusters, application services written in multiple languages, managed databases, message queues, and dozens of supporting components, and each of those systems exposes its own metrics through different mechanisms.

Some metrics come from OpenTelemetry instrumentation embedded in application code. Others come from Prometheus exporters, tiny services designed to expose metrics for systems like PostgreSQL, Redis, Kafka, or MySQL. Kubernetes adds its own signals through tools like kube-state-metrics and cAdvisor, and cloud providers add yet another layer with metrics from services like EC2, Lambda, S3, and API gateways.

Each source uses a slightly different format. Some metrics are defined in YAML metadata files, while others are embedded directly inside source code. Python libraries may use decorators or instrumentation APIs. In many cases the only documentation is a handful of scattered comments in repositories.

Put all that together and the observability ecosystem starts to resemble an archaeological dig. The data exists; finding it is the hard part.

A project that tried to map the chaos

Faced with this fragmentation, one engineer built a tool that feels almost obvious in hindsight: a public metric registry.

Instead of making you search manually through dozens of repositories, the registry scans source code, documentation, and metadata files across observability ecosystems. The result is a searchable catalog containing thousands of metrics from OpenTelemetry, Prometheus exporters, Kubernetes components, and cloud services. The current dataset includes over 3,000 metrics gathered from a wide range of sources.

OpenTelemetry Collector components alone contribute more than a thousand metrics. Prometheus node exporters add hundreds more, and Redis, MySQL, and PostgreSQL exporters each introduce their own sets of telemetry signals. Then there are metrics from AWS CloudWatch covering services like Lambda, DynamoDB, S3, and application load balancers.

Individually, each of these metrics is documented somewhere. Until projects like this appeared, though, they rarely existed in one place, and that's the problem the registry tries to solve.

Planning before you instrument

When engineers first hear about a metric registry, they often assume it's there for documentation. One comment from the community pointed to a more useful case: understanding what metrics would appear if you deployed a certain exporter or instrumentation library, on top of seeing what already exists.

That matters more than it sounds. Observability planning often happens before systems are deployed, and engineers want to know what signals they'll have once monitoring tools are installed. If you deploy the Prometheus PostgreSQL exporter, what database metrics will appear? If you instrument your service using OpenTelemetry, what request metrics will become available automatically? If you enable Kubernetes monitoring components, what cluster health signals can you collect?

Without a central reference, answering those questions means digging through documentation and source code. A registry makes that much simpler.

The everyday pain engineers recognize

The reaction from other engineers was immediate. Many said the scattered nature of observability metrics is one of the most frustrating parts of monitoring systems, with metric definitions spread across countless GitHub repositories, exporter packages, and documentation sites. Even experienced engineers sometimes struggle to track down the meaning of a specific metric name.

One developer put it simply: pulling Prometheus and OpenTelemetry metrics information into a single place makes it far easier to plan monitoring strategies before deploying new infrastructure.

That comment hints at something bigger. Observability has matured dramatically over the last decade. Tools for collecting telemetry are powerful, and storage systems can handle enormous volumes of data. Understanding the signals themselves, however, is still surprisingly difficult.

Why observability documentation still feels fragmented

Part of the problem lies in how observability ecosystems evolved. Most exporters and instrumentation libraries were created independently by different communities, and each project documented its metrics in whatever format made sense at the time. Some use structured metadata files, others embed descriptions directly in source code, and some rely on auto-generated documentation. Very few follow a universal standard.

Even OpenTelemetry, which aims to standardize telemetry across languages and frameworks, still contains many separate instrumentation packages maintained by different teams.

Nobody is really to blame for that fragmentation, but it still creates a discovery problem for engineers trying to understand their monitoring systems.

The scale of modern metrics

One detail in the registry project shows how quickly metrics multiply. The Prometheus node exporter alone exposes over five hundred system metrics covering CPU usage, memory pressure, disk I/O, and network activity. Kubernetes monitoring components add hundreds more signals about pods, nodes, and cluster state, and application instrumentation adds request metrics, latency distributions, error rates, and custom business signals.

It's easy for a moderately complex system to produce thousands of metrics. Most engineering teams actively monitor only a small fraction of them, and the rest stay hidden until something goes wrong.

Observability's next problem: discoverability

For years the observability industry focused on three technical challenges: how to collect telemetry, how to store it, and how to visualize it. Those problems are largely solved.

The next challenge might be discoverability. Engineers need better ways to understand the telemetry their systems can generate, with tools that explain which signals exist, where they originate, and what they mean. Metric registries are an early step toward that goal, turning observability from a guessing game into something closer to a searchable knowledge base.

The shift toward observability knowledge systems

As observability ecosystems grow more complex, tools that organize telemetry knowledge may become just as important as the tools that collect it. Engineers may start treating metrics as structured information instead of isolated signals: where a metric originated, what system emitted it, what it measures, and what alerts typically rely on it.

Answering those questions quickly could make debugging faster and monitoring strategies more effective, and it might save engineers countless hours digging through repositories.

What the registry shows

At first glance, a searchable catalog of metrics might sound like a niche project. It says something about modern infrastructure, though: we've become incredibly good at generating telemetry and are still learning how to understand it.

Until engineers can easily answer the question "what metrics does my system produce," observability will remain part science, part detective work.