What is the difference between automatic, semi automatic, and manual remediation in IT operations?

Automatic, semi automatic, and manual remediation differ mainly in who is allowed to authorize and execute the recovery action. Automatic remediation runs a predefined low-risk action without waiting for a human at execution time. Semi automatic remediation lets the platform diagnose and prepare or perform part of the response while keeping an authorization or controlled workflow step. Manual remediation keeps the final operational action with a human engineer.

The source design uses these levels because production infrastructure has different risk classes. It explicitly rejects the idea that every incident should become unattended self-healing.

What is automatic remediation?

Automatic remediation is a predefined response that the platform can execute directly when an approved condition is met.

The v3.2 SRE model calls its lowest-risk class L1.

L1 includes known transient events, automatic self-recovery, automatic closure and archiving, and a restricted rule library.

The example interface says the L1 rule library contains 24 classes. That number is part of the example product design, not a general target.

The important requirement is that automatic remediation is limited to known, repeatable conditions. The action and expected result are already understood.

What is semi automatic remediation?

Semi automatic remediation sits between recommendation and fully unattended execution.

The source internal-share deck describes remediation as automatic, semi automatic, and manual, with execution actions triggered after authorization.

A semi automatic flow can look like this:

Platform detects the incident.

Platform identifies the likely cause.

Platform matches a runbook.

Platform proposes the action.

A human authorizes it.

Approved automation executes.

The result is written back.

Recovery is verified.

The human does not need to perform every technical step manually, but the system also does not act without the required authorization.

This is useful for controlled-risk operations.

What is manual remediation?

Manual remediation means a human performs the critical operational action.

The platform can still support the engineer with diagnosis, root-cause evidence, runbooks, work orders, remote access, audit, and validation.

Manual does not mean unsupported. It means the final action requires human execution or physical work.

The source internal-share hardware incident gives a clear example. Marking a node unschedulable and evicting containers can be automated. Generating the repair work order can be semi automatic. Replacing the physical GPU card is manual.

The remediation mode follows the nature and risk of the action.

How does the source v3.2 risk model relate to these three levels?

The v3.2 SRE model uses three risk tiers.

L1: no handling required by a person.

Known transient conditions self-heal and close automatically.

L2: controlled risk.

Approved runbook scripts can execute automatically within strict guardrails, with high-risk command blocking and automatic rollback.

L3: risk-bearing change.

The platform creates the plan, but execution goes through change workflow, two-person approval, canary batches, and full audit.

L3 is explicitly forbidden from fully automatic execution.

These tiers are not exactly the same labels as automatic, semi automatic, and manual, but they express the same operating principle: higher risk requires more human control.

When should automatic remediation be used?

Use it for conditions that are well understood, low impact, highly repeatable, easy to verify, safe to reverse or retry, and covered by an approved rule or runbook.

The source L1 examples are known transient problems.

That is the right place to begin.

Automatic remediation should not be introduced first for rare high-impact failures.

The team should prove the detection, action, and validation loop on predictable cases.

What controls should automatic remediation have?

Automatic does not mean uncontrolled.

The source design still requires an approved rule library, execution records, recovery validation, archiving, and a risk boundary.

For controlled scripts, the source also uses dangerous-command blocking and rollback.

The platform should know what normal recovery looks like.

If the action does not restore the expected state, the incident should leave the automatic path and escalate.

A self-healing rule that retries forever can hide a serious recurring problem.

When should semi automatic remediation be used?

Use semi automatic remediation when the technical action is repeatable but the decision to execute still needs human accountability.

Examples include isolating a production node, restarting a sensitive service, changing a routing state, executing an approved repair script, or reallocating resources with service impact.

The source platform keeps authorization in front of execution actions and preserves audit.

This is especially useful while an organization is building trust in automation.

The platform can remove most manual work without removing the approval point.

What does the human approve in semi automatic remediation?

The human should approve the actual proposed action and target, not a vague ticket.

A good request shows the affected object, likely root cause, evidence, proposed action, target, expected impact, and recovery path.

After approval, the same approved payload should be executed.

The source workflow design supports this by linking approvals with automatic execution nodes and writing results back.

That reduces the gap between what was approved and what actually happened.

When should remediation remain manual?

Keep remediation manual when the action requires physical work, has an uncertain outcome, is difficult or impossible to reverse, carries broad production impact, requires expert judgment, or falls outside the approved automation library.

The source hardware example uses physical GPU replacement as a manual action.

The source L3 model also prevents autonomous execution for high-risk changes.

Manual action is not a failure of automation maturity. It is the correct control when the risk justifies it.

How does physical intervention fit?

Physical infrastructure creates a natural limit to software self-healing.

A degraded accelerator can be detected automatically. It can be removed from scheduling automatically or after authorization. Workloads can be rescheduled. A repair work order can be created.

But replacing the card still requires a person in the data center.

The source explicitly separates those stages.

That is a useful reminder that automated remediation in infrastructure often means automating the response around a physical repair, not eliminating the physical repair itself.

How does rollback affect the remediation level?

Rollback makes controlled automation safer.

The source L2 tier includes automatic rollback on failure.

If a configuration or script can reliably restore the previous state, more of the execution can be automated.

If the action is irreversible, the enterprise should require stronger approval and human control.

A reversible service restart is different from destructive storage action. A configuration change with a saved baseline is different from a physical replacement. A tested deployment rollback is different from an unsupported firmware downgrade.

Reversibility is therefore one input to remediation classification.

How should high-risk commands be handled?

The source L2 tier uses a high-risk command blacklist.

That creates a hard technical boundary inside controlled automation.

Even an approved script should not be allowed to invoke prohibited operations through the automated path.

An enterprise can also use positive allowlists where appropriate.

The source does not prescribe the complete command-control architecture. The source-supported principle is that automated execution should have technical guardrails, not only policy documents.

How should permissions affect remediation?

The execution identity should have only the permissions needed for the approved action.

The source governance layer includes role permissions, sensitive-operation authorization, tenant and project boundaries, and full operation audit.

Automatic remediation should therefore use a scoped service identity.

Semi automatic execution should preserve the requester, approver, and automation identity.

Manual execution should preserve the human operator.

Every level remains accountable.

How does diagnosis differ from execution?

Diagnosis can be more automated than execution.

The source AI assistant and AIOps layers can aggregate alarms, identify likely root cause, show confidence and evidence, estimate business impact, match a runbook, and recommend an action.

Those capabilities do not automatically grant permission to change production infrastructure.

The source repeatedly states that execution actions require authorization.

That separation lets the organization gain value from AI-assisted operations before allowing broad autonomous action.

How should recovery be validated?

Every remediation level should validate the result.

Automatic: the system verifies recovery before closing.

Semi automatic: the system verifies the authorized action and reports the result.

Manual: the engineer completes the action, then monitoring verifies recovery.

The source incident loop explicitly includes recovery confirmation and closure.

Without validation, the platform knows only that an action was attempted. It does not know whether the incident was resolved.

How should failures escalate between levels?

A failed automatic action should move to a higher-control path.

For example, an automatic recovery fails. The incident remains open. A semi automatic runbook is proposed. If that fails or the case requires physical work, a human engineer takes over.

The source does not describe one universal escalation state machine.

But its combination of risk tiers, work orders, exception handling, and manual intervention supports this progressive-control model.

The automation should fail safe.

How should the three levels be measured?

Measure each according to its intended outcome.

Automatic remediation should track success rate, MTTR, recurrence, and operator interventions avoided.

Semi automatic remediation should track approval time, execution success, rollback, manual takeover, and MTTR.

Manual remediation should track time to assignment, action time, repair duration, repeat failure, and vendor response where relevant.

Then measure the combined service outcome through SLO, error budget, and total recovery time.

For the measurement framework, how IT teams can measure whether automated remediation is actually improving operational efficiency explains how automation share and toil should be interpreted.

Why is 100 percent automated a bad target?

Because the source deliberately keeps the highest-risk class outside autonomous execution.

The automation objective should be to automate what is safe and repetitive, assist what is controlled but consequential, and keep humans in charge of what is high-risk, uncertain, or physical.

For the broader governance boundary, how enterprises automate data center operations while keeping approvals, permissions, rollback, and audit controls explains how execution remains accountable at every automation level.

A platform example that uses automatic, semi automatic, and manual response levels within one closed-loop incident process is Sensaka.

If I were setting remediation policy, I would classify the action before deciding the technology. Ask how repeatable it is, how reversible it is, how large the blast radius is, and how much judgment is required. Low-risk known actions can become automatic. Controlled actions can become semi automatic. High-risk or physical actions should remain human-controlled.