Infrastructure Monitoring Ledger Methodology: How Every Score Is Built

Part of the Infrastructure Monitoring Ledger (edition 2026-10-04). The results are in the overview and the vendor reports. The Ledger uses the same method as the VM Platform Ledger and the two backup Ledgers; this page covers what is specific to infrastructure monitoring.

Who this is for

IT operations teams, network operations centres and system integrators choosing or recommending monitoring for enterprise data centres and networks. Many are leaving VMware and need monitoring that covers the new hypervisor. No vendor was briefed on, paid for or reviewed any score.

What is in scope

A product is included when it shipped a release in the last 12 months, has public documentation or release notes, can run self-hosted or on-premises with no mandatory SaaS control plane, and monitors both servers and network devices (SNMP at minimum) out of the box. SaaS-only observability platforms (Datadog, Dynatrace, New Relic, Splunk Observability, Grafana Cloud, LogicMonitor and others) are left for a separate Cloud Observability Ledger. Log-only tools, desktop RMM tools and single-vendor device managers are out of scope.

The 20 entries

Tier	Entries
Leader incumbents	SolarWinds Observability Self-Hosted, ManageEngine OpManager, Zabbix, Paessler PRTG
Legacy enterprise	Nagios XI, Broadcom DX NetOps and DX UIM, IBM SevOne and Netcool, Microsoft SCOM
Challengers	Checkmk, ScienceLogic, Centreon, Icinga, OpenNMS, Grafana Labs self-hosted stack, Netdata, Elastic Observability
Mid-market	Progress WhatsUp Gold, Opsview
Community project	LibreNMS
Open-source engine	Prometheus

Commercial products, open-core platforms and open-source projects sit on one scale. For an open-source project the Ledger scores the project itself: a capability that needs a third-party exporter, plugin or add-on counts half at most, and the report names the add-on.

Capability: ten domains, scored since 2016

Domain	Weight	What a 3 (solid) means	What a 5 (reference) means
Server, virtualization and storage	12	Windows and Linux agents and agentless checks, VMware and Hyper-V, basic storage-array health	Broad hypervisor list, storage-array packs, hardware health, containers and Kubernetes nodes, auto-discovery at scale
Network monitoring	12	SNMP polling and traps, interface metrics, templates for major vendors	Flow analysis, topology discovery and maps, configuration management (NCM), wireless, IPAM
Application and service monitoring	10	Service checks, process and Windows service monitoring	Synthetic transactions, dependency mapping, database performance, OpenTelemetry traces, business-service views
Data collection and scale	12	Agents and agentless collection, remote pollers	Distributed polling, server high availability, documented large scale, long-term storage
Alerting and event management	12	Thresholds, dependencies, escalations, notifications	Rule- and topology-based correlation and deduplication, root cause on topology, maintenance windows, on-call integrations
Visualisation and reporting	8	Dashboards, maps, availability and SLA reports	Custom dashboards and maps, capacity trend reports, multi-tenant reports, business-service views
Logs and traces	8	Syslog and Windows event logs with alerting	Full log ingestion and search, OpenTelemetry logs and traces, correlation with metrics
Automation and remediation	8	Actions on alert, REST API	Runbook automation, closed-loop remediation, ITSM integration, configuration as code
Security and compliance	8	RBAC, MFA or SSO, encrypted agent traffic	Multi-tenant isolation, audit trail, hardened and FIPS-capable builds, secrets vaulting, air-gapped operation
New-platform support (VMware exit)	10	Monitors at least one non-VMware, non-Hyper-V hypervisor out of the box	Native templates for Proxmox, Nutanix AHV, XCP-ng, OpenShift Virtualization, OpenStack, Kubernetes and public cloud

Each domain is scored 0 to 5 at the end of 2016, 2019 and 2022 and today, against the same anchors every year, so a score only rises when something ships. Only released capability counts; previews, betas and 0.x versions count half a step at most. The ten weighted scores add up to capability out of 100. Buyer lenses on the interactive page re-weight the domains for a VMware leaver, a network operations centre, enterprise IT operations, an MSP, a regulated or air-gapped site, and an open-source-first team.

No double counting with AI. Rule-based and topology-based features score in the capability domains. Machine-learning features (dynamic baselines, anomaly detection, machine-learning correlation, forecasting) score only in AI leverage.

The security record of the product itself

Monitoring servers hold credentials for everything they watch, so a monitoring product's own security record matters more than in most categories. For each flaw in the scored product that is on the CISA Known Exploited Vulnerabilities catalogue with a catalogue date in 2023 to 2026, today's security score falls by 0.5, up to 1.5. A supply-chain compromise of the product itself, as with SolarWinds SUNBURST in 2020, lowers the security score by 1.0 in the year it was disclosed; it recovers only on documented changes to how the product is built. Only the scored monitoring products count: flaws in a vendor's other products are listed in the report as context.

The other dimensions

Credibility (out of 25): release cadence, roadmap transparency, the say-do record (what was announced in 2023 to 2026 and whether it shipped), velocity and lifecycle stability.
Commercial risk (out of 25, lower is better): price and licence changes, purchase constraints, channel access, owner stability and the cost of staying supported. Every vendor that sells the product or support is scored this way, including the companies behind Zabbix, Checkmk, Icinga, Centreon, OpenNMS, Grafana, Netdata and Elastic.
Sustainability risk (out of 25, lower is better): used instead for LibreNMS, which has no company behind it: maintainer concentration, backer health, licence-change risk, abandonment signs and security response.
Prometheus is an open-source engine with nothing to buy: it has no commercial-risk score, and its buyer safety uses lock-in only.
Technical lock-in (out of 25, lower is better): how open the configuration and history formats are, whether configuration and metric history can be exported, coupling to the vendor's own agents, database or appliance, hardware or cloud coupling, and skills.
Integrator fit (out of 20): partner programme, multi-tenancy and self-service, API and infrastructure-as-code, skills and certification.
AI leverage (out of 20): AIOps on monitoring data, AI services built into the product, AI-assisted operations, and official agent interfaces such as MCP servers. Community MCP servers do not count.
Momentum: capability gained since the end of 2022.

The Ledger Index

A single number out of 100: capability 35, credibility 20, buyer safety 20, AI leverage 10, integrator fit 10 and momentum 5. Buyer safety is one minus the sum of commercial (or sustainability) risk and lock-in, divided by 50. The Index is shown only from 2023, because credibility, risk and fit are scored for 2023 to 2026; earlier years show capability and AI only. The interactive page offers three other weightings: risk-averse buyer, innovation-first and integrator.

Evidence and calibration

Every fact behind a score comes from public sources: release notes, documentation, changelogs, GitHub and GitLab release and tag dates, vendor press releases, regulatory filings and the CISA catalogue. Release dates for the open-source projects were checked automatically against their repositories. Scores were drafted against the anchors and then calibrated side by side across all 20 entries, so the same evidence earns the same score whichever vendor it belongs to; a "5" is kept for the reference implementation in a domain. Each report carries an evidence grade (A for primary sources, B where sources conflict or rely on vendor statements, C for inference) and lists its open questions.

Archetypes

Each report ends with a short archetype label, such as "Open-source workhorse" or "Broad incumbent, private-equity owned". These are readings of the scores and the release record, not extra scores.

Limits

The Ledger measures what was shipped and documented, not how well it runs in a particular environment. Vendor statements about scale and performance are labelled as such. Prices are rarely public in this category, so commercial risk relies on licence models and dated changes rather than list prices. Market share and popularity are deliberately not scored.