Who's accountable when an AI-driven health insurance denial affects patient care?

The shift

Producing a coverage or medical-necessity determination goes from scarce reviewer time — a nurse or medical director reading a chart against policy — to abundant: a model ingests the record and the plan's criteria and emits a denial in seconds, at a volume no human panel could match. Review capacity stops being the thing that limits how many denials an insurer can issue.

The axioms

• The number of claims and prior-auth requests an insurer can deny is gated by scarce, expensive human review capacity (nurses, medical directors, hours in the day).
• A denial is a decision that a named, licensed party is legally and regulatory accountable for.
• The treating physician's clinical judgment about what the patient needs is a scarce, authoritative input to the coverage call.
• The patient (and physician) can appeal a denial, and the appeal exists to catch bad calls before they cause harm.
• The insurer owes a good-faith duty to actually review the case before denying — a denial implies a review happened.
• Regulators can inspect a denial and hold a specific decision-maker to account, because there's a reviewable record of who decided and why.

Invalid axioms

1. The volume of denials is gated by scarce human review capacity. Reading a chart against policy criteria and generating a defensible-looking denial rationale is now near-free and instant. The habit-trap: regulators, plan contracts, and the insurer's own compliance functions are still built on the assumption that denial volume is naturally throttled by how many reviewers you can staff — so denial rates per reviewer, turnaround-time rules, and audit sampling all calibrate to a human-paced pipeline that no longer exists. When one "reviewer" can nominally sign thousands of AI-generated denials a day, the staffing-based checks that used to slow bad denials down stop biting.

Unchanged axioms

1. A named, accountable party is legally and regulatorily answerable for a denial. Liability doesn't transfer to a model. When a denial harms a patient, ERISA fiduciary duty, state bad-faith law, and insurance regulators still attach to the insurer and, where required, to the licensed medical reviewer who signed. A model cannot be a fiduciary or hold a license, so this doesn't move as models improve — it's a legal fact, not a capability gap. What is moving fast, and worth flagging: several states are legislating that AI cannot be the sole basis for a medical-necessity denial and that a licensed clinician must make the call. Those statutes are the current firewall; their reach and enforcement are unsettled as of mid-2026.
2. The treating physician's clinical judgment is an authoritative input the model can't replace. The physician has examined the patient, carries the malpractice liability for the treatment decision, and can assert facts about the case that aren't in the record the model read. A denial that overrides that judgment is overriding an accountable human with a model that answers to no one.
3. The patient has a right to appeal, and the appeal is where a wrong denial is supposed to be caught. Internal appeal and external independent review are legally mandated backstops. Cheaper generation of the initial denial doesn't remove the right; if anything it raises how much weight the appeal has to carry.
4. The insurer owes a good-faith duty to actually review the case before denying. A denial is a representation that the case was evaluated on its merits. That duty is a legal obligation, not a throughput target, and it doesn't get cheaper because generating the denial did.
5. Being confidently wrong causes direct harm here. A plausible-but-wrong denial delays or blocks care for a specific patient — a missed treatment window, a rationed drug, a forgone procedure. The cost is not a redo; it's asymmetric and physical, which raises the bar on verification rather than lowering it.

New axioms

1. When denials can be generated at machine speed, "a licensed reviewer signed it" no longer proves a review happened. If a clinician approves denials faster than anyone could read the underlying cases, the signature is nominal and the good-faith-review duty is met on paper only. We need a way to distinguish a real medical-necessity review from an automated one wearing a human signature — and audit methods calibrated to human throughput can't see the difference.
2. Who is liable when an AI-driven denial harms a patient, and how is that apportioned? The vendor that built the model, the insurer that deployed it, the reviewer who signed, and the plan sponsor all touch the decision. Bad-faith and fiduciary law assumed the deciding mind and the accountable party were the same. When the denial was substantially produced by a model and a human rubber-stamped it, the doctrine for splitting liability is unsettled — and litigation, not statute, is currently defining it.
3. Automation bias turns the appeal into a rubber-stamp of the original model. If the appeal reviewer sees the AI's rationale first, or if the appeal itself is triaged by the same class of model, the mandated backstop collapses into a second pass of the thing it was supposed to check. Independence of review has to be engineered against the model, not just against the original human reviewer.
4. The clinician now argues with an opaque model instead of a peer. Peer-to-peer review assumed the physician could reach an accountable clinician and reason with them. When the denial rationale is model-generated and the criteria are proprietary and non-inspectable, the physician is contesting a black box with no one who can actually defend or revise the specific call. We need a right to an intelligible, contestable rationale — not a generated paragraph that cites criteria the physician can't see.

Where it breaks

"Denial volume is gated by human review capacity" (invalid) collides head-on with "the insurer owes a good-faith duty to actually review before denying" (still holds). The good-faith duty was enforced, in practice, by the fact that reviews were slow and expensive — you couldn't deny at scale without staffing the reviews, and the staffing left an audit trail. Remove the capacity constraint and the duty still legally stands, but nothing operational enforces it: a plan can issue denials far faster than any genuine review could occur while every one carries a compliant-looking signature. The firewall is now the emerging "a licensed human must decide" statutes — but those are enforced by the same human-paced audit assumptions the flip just invalidated.

A second collision: "the patient can appeal and the appeal catches bad denials" (still holds) meets "automation bias turns the appeal into a rubber-stamp" (new). The appeal is doing more load-bearing work than ever, precisely when it's most exposed to being contaminated by the same model whose output it's supposed to independently check — and no one has decided who is accountable for keeping the appeal genuinely independent of the model that generated the denial.

ai-axioms.vercel.app/axioms/healthcare/healthcare-insurer-denial-ai-axioms