Who's accountable when predictive-policing and AI surveillance drive an arrest, and does it launder bias?

The shift

Deciding where and whom to police — patrol allocation, target selection, watchlists — goes from scarce human attention to abundant, continuous, population-wide prediction; and matching a person against surveillance feeds (face, plate, location history, behavioral pattern) goes from expensive investigation to near-free and instant. What stays exactly where it was is the constitutional floor: an arrest still requires individualized suspicion a named official can defend, and a person harmed by a wrong one is still owed a basis they can contest.

The axioms

• An arrest rests on individualized suspicion — particular facts about a particular person, not membership in a category. Rests on forming suspicion being costly, so it attaches to concrete observed facts about one person.
• A specific officer decides to detain and is identifiable as the author of that decision. Rests on the decision-maker and the accountable party being the same human by default.
• Where and whom to police is set by scarce human attention — patrol hours, detective time, tips. Rests on surveillance and prediction being expensive and therefore selective.
• Recorded crime data is a reasonable measurement of where crime happens. Rests on arrest and incident records being treated as a neutral signal of underlying offending rather than of past enforcement choices.
• Due process lets a person contest the basis of their arrest and confront the evidence against them. Rests on the arrest having a statable, human-authored basis that can be examined in the open.
• The system's legitimacy depends on suspicion being justifiable — to the person stopped and to a court. Rests on trust being tied to defensible, individualized reasons a citizen can hear and challenge.
• A biased outcome is attributable — it traces to a decision-maker who can be identified and held to it. Rests on discretion being human, located in a person or a policy someone signed.

Invalid axioms

1. Where and whom to police is set by scarce human attention. Allocating patrols, ranking neighborhoods by risk, and generating lists of people "worth watching" is now cheap, continuous, and scalable across a whole jurisdiction — the model reads years of records and camera feeds and outputs targets faster than any command staff could. The scarce input that used to force selectivity is gone. The habit-trap: departments treat a model-generated hot-spot map or a person-of-interest list as more objective than an officer's hunch because it processed more data — when the thing that got cheap is producing the prediction, not verifying it corresponds to reality. Volume of inputs gets read as quality of judgment.
2. Recorded crime data is a reasonable measurement of where crime happens. This was a workable-enough proxy when data was sparse and humans read it with local knowledge. At scale it fails on its own terms: arrest and stop records measure where police went, not where crime is, and the two diverge most exactly where enforcement has historically concentrated. Feeding that history to a model that allocates tomorrow's patrols turns a record of past policing into a forecast of future crime. The habit-trap: the output is called a crime prediction and defended as data-driven, when it is more accurately a prediction of where the department will next make arrests given where it made them before — a claim about police behavior dressed as a claim about offenders.

Unchanged axioms

1. An arrest requires individualized suspicion — particular facts about a particular person. A statistical claim that an area is high-risk, or that a person resembles a pattern, is not by itself a fact about what this person did. Courts still require the suspicion to be particular and articulable, and "the algorithm flagged them" is a category-membership claim, not an individualized one. This didn't get cheaper or weaker; the flood of cheap predictions makes it more load-bearing, because more encounters now begin from a model output that has to be converted into real individualized grounds before a lawful arrest can follow. (Whether courts hold this line as model outputs get more granular and persuasive is a fast-moving call — the doctrine is being litigated now, not settled.)
2. A person can contest the basis of their arrest and confront the evidence. Due process presupposes a basis that can be stated and examined. That right is intact as written; the question the shift raises is whether it can be honored, not whether it applies. The obligation to disclose and defend the grounds sits with the state regardless of what produced them.
3. Legitimacy depends on suspicion being justifiable to the person and a court. Legitimacy is a relationship between the policed and an authority they can hold to account, not a property of the model's accuracy. A stop that can't be explained in terms the person and a judge can test corrodes legitimacy even if the underlying prediction was, in aggregate, statistically sound — and the communities most exposed to the tools are the ones whose consent is already thinnest. This is where over-deployment does its slow damage regardless of hit rate.
4. An accountable human must decide to detain and own that decision. A model can flag, rank, and match; it cannot be answerable, cannot be sued, cannot testify to its reasons under oath. The legal duty to justify a seizure lands on an officer and a department. Cheaper prediction doesn't relocate that duty — it makes it easier to obscure who actually exercised the discretion, which is a different problem, handled below.

New axioms

1. Historical enforcement bias is laundered into a neutral-looking risk score, and the objection loses its target. When a person or place is flagged, the disparity that was visible as "officers stop more people in this neighborhood" becomes "the risk model ranked this neighborhood high." The bias didn't leave; it moved upstream into the training data and out of view, and acquired the authority of math. The old objection — you are policing us because of who we are — now has to be made against a system that returns a number, and the number's provenance is a proprietary model trained on records the person can't see. Fairness has to be adjudicated on data and design the accused has no access to.
2. A feedback loop manufactures the evidence for its own predictions. Send more patrols where the model predicts crime, and you record more stops and arrests there, which becomes training data confirming the area is high-crime, which sends more patrols. The loop is self-validating: the prediction looks accurate because it caused the enforcement that generated the data proving it right. Departments must solve for how to measure a model's correctness when the model shapes the reality it is scored against — ordinary accuracy metrics are complicit in the loop.
3. Accountability for an arrest diffuses across officer, department, and vendor with no one holding the whole of it. The officer says the system flagged the target; the department says it deployed a validated tool and the officer made the call; the vendor sells "decision support," disclaims liability, and shields the model as a trade secret. The duty to justify the seizure still lands on the state, but the internal locus — whose judgment, whose bias, who answers to the person arrested — has no default owner the way a named officer's articulable hunch once did. (Proponents argue the tools can reduce arbitrary discretion and document decisions better than an unrecorded hunch; that upside is real only if the model's basis is auditable and contestable, which is exactly what trade-secret deployment prevents.)
4. Appealing an algorithmic suspicion has no venue. A person can contest an officer's stated reasons in court. There is no equivalent process for "you were on a list," when the list is generated by a model whose inputs, weights, and error rate are undisclosed, sometimes even to the officers using it. The right to confront the evidence assumes the evidence can be produced; a proprietary risk score resists production. Government must solve for what confronting the evidence means when the evidence is a model output nobody in the courtroom can open.

Where it breaks

Departments adopt cheap population-wide prediction to allocate policing (INVALID #1) on data that measures past enforcement rather than crime (INVALID #2) — while an arrest still legally requires individualized, contestable suspicion (STILL HOLDS #1, #2). The break: the model's output is a category-membership claim ("this person/place matches a high-risk pattern"), and the law requires an individualized one, so officers convert the flag into individualized grounds after the flag directed them there — the stop that generates the articulable facts was itself caused by the prediction. The suspicion looks individualized on the arrest report and was population-level in origin, and the report is the only part the court sees.

A second collision: the legitimacy the system needs (STILL HOLDS #3) and the right to confront the basis (STILL HOLDS #2) meet bias laundered into an unauditable score (NEW #1) with no venue to appeal it (NEW #4). The disparity that was once contestable as visible discretion is now defended as objective analytics and shielded as a trade secret — so precisely when the grounds for a stop most need to be explainable and testable, they become least available to the person stopped and to the court. Whether an opaque risk score can satisfy the right to confront the evidence is being answered by default, deployment by deployment, before it is answered by law.

ai-axioms.vercel.app/axioms/government/policing-predictive-surveillance-ai-axioms