No. 49 / 339
What shifts in accountability when an autonomous agent, not a human, executes the remediation?
The shift
Diagnosing a likely cause and running the fix — the mechanics of remediation — goes from something only a scarce, credentialed human could do (with unavoidable latency and cost) to something an agent can do instantly, at 3am, in parallel across a whole fleet. What doesn't move: the agent can't be fired, sued, or lastingly distrusted. Actor and accountable party used to be fused by necessity — only a human could act, so whoever acted was who you held to account. That fusion breaks now, and nothing has stepped in to replace it.
The axioms
- Whoever has hands on the keyboard during remediation is whose judgment gets scrutinized afterward. Rests on scarcity of who could act — only a credentialed human with access, so actor and decision-maker were the same person.
- Executing a runbook step is itself a judgment call, even when it's copy-paste. Every remediation action cost human attention, so choosing to run it was evidence someone thought it was right.
- Blameless postmortems work because the person who acted can narrate their reasoning, intent, and what they saw at the time. Rests on scarcity of introspectable intent.
- On-call rotations exist because at 3am there's exactly one scarce human awake who has to decide alone, right now.
- Liability lands on a person or their manager because organizations need a locus that can be fired, sued, apologize, or be trusted less next time. Accountability requires a party capable of feeling consequence.
- Runbooks are written for humans who can notice when reality doesn't match the script and deviate. Rests on scarcity of judgment under distribution-shift — pattern-matching plus the sense to know when the pattern doesn't apply.
- Trust in a fix scales with the engineer's track record — a senior's remediation is trusted more because of demonstrated judgment over time. Rests on scarcity of reputation built through repeated, attributable decisions.
Invalid axioms
- The person who acted is who gets scrutinized. Speed and availability of execution were scarce; they're not anymore. An agent can run the fix in the time it takes a human to open the alert. Habit-trap: postmortem templates still ask "what did the responder do" as the first question, assuming the responder made the call, when increasingly they approved — or didn't even see — what the agent already ran.
- Running a remediation step is itself evidence of a judgment call. When a human executed, the friction of doing so filtered out bad ideas — nobody bothers to run a fix they don't half-believe in. An agent has no such friction; it executes anything that scores well against its policy, at zero personal cost to itself. Habit-trap: treating "the system ran it" as equivalent to "someone judged it sound," and logging agent actions the same way as human ones in the incident timeline.
- On-call rotations exist to guarantee a scarce human is available to decide. Availability of competent, instant response is no longer scarce at 3am. Habit-trap: still staffing and paying for someone to be "on the hook" primarily for their presence, when their actual job has shifted to reviewing what already happened.
Unchanged axioms
- Liability needs a locus that can be fired, sued, or lose standing. An agent cannot be held accountable in any sense that changes its future behavior through consequence — it doesn't fear, doesn't lose reputation, doesn't carry risk. Whoever configured, deployed, or approved the agent's authority to act absorbs that role by default, whether or not they intended to.
- Judgment under distribution-shift — knowing when the runbook doesn't apply — stays scarce. Agents remediate well against patterns they've seen; the incident that doesn't match any pattern (novel failure mode, ambiguous blast radius, conflicting signals) is exactly where confident-but-wrong is most dangerous and most likely.
- Narratable intent for a postmortem stays a human property. An agent can produce a plausible-sounding rationale for its action after the fact, but that rationale is generated to fit the outcome, not a reliable record of what it "believed" at decision time the way a human's account is (imperfect as that is too). Blameless-postmortem culture assumed an account you could interrogate and calibrate trust against.
- Trust built through track record doesn't transfer to a fleet of agent runs. Reputation was a signal about a specific person's judgment, accumulated slowly, applied to future ambiguous calls. An agent's success rate on past incidents is a different kind of evidence — it says nothing about whether this incident falls inside its competence, and treating a good track record as a proxy for trustworthiness on a novel case is a category error that's easy to make.
New axioms
- Blast radius now scales with execution speed, not human attention span. When a human executed, a bad fix propagated at the pace of one person doing one thing. An autonomous agent can apply the same bad fix across every affected host before anyone notices the first one was wrong. Incident response has to solve for containment and rollback of agent action at a speed nothing currently reviews at.
- Approval is becoming a rubber stamp that still carries full accountability weight. If a human "approves" an agent's proposed remediation without truly understanding it — because there are ten of these a shift and each looks plausible — they're accepting legal and organizational accountability for a decision they didn't actually make. Nobody has settled whether that approval is meaningful consent or theater, and insurance/liability frameworks haven't caught up either way.
- Audit trails are growing faster than anyone's capacity to review them. Every agent action is logged, which sounds like more accountability, not less — but volume of record isn't the same as anyone actually being answerable for what's in it. The bottleneck moves from "did we capture what happened" to "does anyone with authority ever look."
- The line between "agent suggested" and "agent executed" is where all the risk concentrates, and orgs are drawing it inconsistently. Some remediation flows keep a human in the loop for execution; others let the agent close the loop autonomously once confidence crosses a threshold. That threshold is currently a product/engineering decision, not a governance one, even though it's the actual accountability boundary.
Where it breaks
Postmortems still ask "who approved this fix" as if the answer settles accountability (invalid) — but when approval is a rubber-stamped glance at the tenth plausible-looking agent action that shift (new), the named approver is accountable on paper for a judgment they never actually exercised. The org gets a name to point to and no real improvement in whether the next bad fix gets caught.
On-call is still staffed and paid as the scarce-availability role (invalid), while blast radius from autonomous execution now scales faster than any human review loop (new) — the person nominally "on the hook" is often structurally unable to intervene before an agent's fix has already propagated fleet-wide, making their accountability nominal rather than functional.
Related axioms
Engineering
What changes for data engineering with AI?
Engineering
What changes for DevOps with AI?
Engineering
What changes for hardware engineering with AI?
Engineering
What changes for ML engineering with AI?
Engineering
What changes for QA and testing with AI?
Engineering
What changes for software engineering with AI?
Other axioms
Media
What changes for the creator economy (influencers, podcasters, streamers) with AI?
Education
What changes for vocational education with AI?
Government
What changes for intelligence analysis with AI?
Industries
What changes for energy with AI?
Research
Is the paper still the right unit of scientific output when AI can generate them faster than humans can read them?
Research
Is hypothesis generation still a scientist's job when AI systems can propose and rank novel hypotheses themselves?