Infrastructure Automation Ledger Methodology: How Every Score Is Built

Part of the Infrastructure Automation Ledger (edition 2026-10-16). The results are in the overview and the tool reports. The Ledger uses the same method as the VM Platform Ledger; this page covers what is specific to automation tools.

Who this is for

Infrastructure and platform teams that provision and configure servers, virtual machines, hypervisors, network gear and cloud accounts, and the system integrators who build practices on these tools. Many readers run on-prem or hybrid estates, so every report records whether a maintained provider, module or plug-in exists for vSphere, Proxmox VE, Nutanix, Hyper-V, XCP-ng and KubeVirt, who maintains it and when it last shipped. No vendor was briefed on, paid for or reviewed any score.

What is in scope

Three families share the Ledger: infrastructure-as-code engines and the platforms that run them, configuration management, and host or bare-metal provisioning. A tool is included when it shipped a release in the last 12 months, has public documentation or release notes, is available outside one country, and manages infrastructure declaratively or idempotently with a plan, dry run or noop mode and a way to see drift.

Three readings of that last test are made explicit:

Bare-metal provisioning tools (MAAS, Ironic, Metal3, Tinkerbell) keep desired-state records and reconcile or report state, but have no plan mode. They are included and each report says so. Cobbler, which has none of these, is out.
Run platforms (Semaphore UI, Terragrunt) pass the test by running Terraform, OpenTofu or Ansible rather than by their own planning. They are included with a scope note.
Atlantis is scored but not ranked, because its drift detection is an alpha API behind a flag.

Out of scope: CI/CD servers, Kubernetes application GitOps controllers (Argo CD, Flux), image builders on their own, secrets managers, and tools with no release since October 2025 (AWX, last released 2024-07-02). Rundeck and StackStorm are job and event engines without a plan or drift mode and are out. System Initiative archived its repository in February 2026 and is out. AWS CloudFormation and Azure Bicep are kept as single-cloud reference points: they show what a native tool offers and score low on provisioning breadth and high on cloud coupling by design.

Engines, products and community projects

ansible-core is the engine inside Red Hat Ansible Automation Platform and The Foreman is the engine inside Red Hat Satellite; engines carry no commercial risk score, because nobody sells them. OpenTofu, Crossplane, OpenVox, Cinc, Ironic, Metal3 and Tinkerbell are community projects that no company sells either, but they are products in their own right, so they carry a sustainability risk score in place of commercial risk: maintainer concentration, backer health, licence-change risk, abandonment signs and security response.

Capability: ten domains, scored since 2015

Domain	Weight	What a 3 (solid) means	What a 5 (reference) means
Provisioning breadth	15	Maintained providers or modules for the major clouds plus at least two on-prem platforms (vSphere, Proxmox VE, Nutanix, Hyper-V, XCP-ng, KubeVirt), some vendor or partner maintained	Vendor-verified coverage across the major clouds and all the on-prem platforms above, plus network, storage and SaaS, with a large registry
Configuration management depth	12	OS, package and service configuration, idempotent runs, Linux and Windows, agent or agentless, drift visible	Idempotent convergence at fleet scale, continuous drift detection with automatic remediation, Linux, Windows, network and mainframe, compliance content
State, planning and safety	12	Plan or preview, state or inventory storage with locking, import of existing resources, a policy check possible in CI	Saved and approved plans, remote state with locking and encryption, import and refactoring moves, a built-in policy gate before apply, drift detection
Language and authoring	10	A documented language, a module, role or cookbook registry, basic IDE support, some test tooling	A typed language or strong DSL, a native test framework, a registry with signed content, a first-class language server
Orchestration and self-service	10	Dependency graphs or ordered runs, scheduling, a web UI or API to launch runs, role-based launch	Approvals, event-driven runs, workflow chaining across tools, a self-service catalogue with guardrails, multi-tenancy
Scale and control plane	8	Thousands of nodes or resources, parallel runs, an HA option for the control plane	Tens of thousands of nodes or very large stacks, a horizontally scaled control plane, agent fleets, documented air-gapped operation
Security and compliance	10	Encrypted secrets, RBAC, an audit log, single sign-on in at least one edition	SAML and OIDC, fine RBAC, policy as code, a full audit trail, FIPS, signed artefacts and provenance
Bare metal and day-2 operations	8	Drives PXE, kickstart, preseed or image deploys, or patches and lifecycles hosts, in a basic way	Full host lifecycle: discovery, PXE and image provisioning, BIOS, firmware and BMC, patching, decommission, at scale
Ecosystem and integrations	8	CI/CD, Git, ITSM and secrets integrations, an active community content base	Deep integrations with CI/CD, GitOps, ITSM, observability and marketplaces; a very large, healthy content base
AI assistance	7	GA AI generation or explanation of code and plans inside the product, or an official MCP server GA	GA generation plus plan explanation plus a GA official MCP or agent interface with identity, guardrails and audit

Each domain is scored 0 to 5 at the end of 2015, 2019 and 2022 and today, against the same anchors every year, so a score only rises when something ships. Only generally available capability counts; previews count half a step at most. Where a product line changed (Terraform Cloud to HCP Terraform, Open Source Puppet to Puppet Core, Digger to OpenTaco), each year scores what a customer could get then. Buyer lenses on the interactive page re-weight the domains for an on-prem estate leaving VMware, a cloud-first platform team, a regulated or air-gapped site, bare-metal and edge fleets, and an open-source-first team.

The other dimensions

Credibility (out of 25): release cadence, roadmap transparency, the say-do record (what was announced in 2023 to 2026 and whether it shipped), velocity and lifecycle stability. Licence changes, source moves and forks count against lifecycle stability.
Commercial risk (out of 25, lower is better): price and licence changes since 2023 (the Business Source License on Terraform, Puppet's move to private binaries, Chef's licensed downloads), purchase constraints, channel access, owner stability (acquisitions by IBM, Perforce, Progress and Broadcom) and the cost of staying supported.
Sustainability risk (out of 25, lower is better): used instead for community projects, as described above.
Technical lock-in (out of 25, lower is better): state and code formats, the export path to another tool (Terraform to OpenTofu, Puppet to OpenVox, Chef to Cinc), coupling to the vendor's platform or cloud, and skills.
Integrator fit (out of 20): partner programme, multi-tenancy and self-service, API and infrastructure-as-code depth, skills and certification.
AI leverage (out of 20): AI code and plan generation, bundled AI platforms, AI assistants for operators, and official agent interfaces such as MCP servers. Only GA counts, and each report lists the AI items announced since 2023 and how many reached GA.

The Ledger Index

A single number out of 100: capability 35, credibility 20, buyer safety 20, AI leverage 10, integrator fit 10 and momentum 5 (capability gained since the end of 2022). Buyer safety is one minus the sum of commercial (or sustainability) risk and lock-in, divided by 50; for engines it uses lock-in alone. The Index is shown only from 2023.

Evidence and calibration

Every score comes from the public record: release notes and changelogs, GitHub release feeds and licence files, documentation, lifecycle and pricing pages, and press releases for ownership changes. Reports carry an evidence grade (A primary sources, B secondary, C inferred) and end with open questions. History before 2020 is thinner for the younger platforms and is graded accordingly. Corrections are welcome from anyone, vendors included, with a public source, and are logged in the next edition. The Ledger is refreshed twice a year, in March and October.