What changes for ML engineering with AI?

The shift

Writing and translating ML code — training loops, data pipelines, architecture boilerplate, reproducing a paper's method — goes from scarce specialist labor to an abundant first draft an LLM produces in seconds. What doesn't get cheaper: knowing whether the model is actually right, why it failed, and who owns the consequences when it's wrong in production.

The axioms

1. Writing training/inference code and pipeline boilerplate requires scarce specialized engineering time.
2. Feature engineering and architecture choices require scarce human trial-and-error intuition.
3. Labeled data is scarce and expensive, and gates what's learnable.
4. Deep debugging of model internals (loss spikes, gradient issues, silent shape bugs) requires a scarce pool of engineers who understand the math and systems together.
5. Compute is scarce and expensive, so experiments must be planned carefully before running.
6. Reproducing state-of-the-art results requires scarce access to papers and the expertise to translate them into working code.
7. A human must decide whether a model is good enough to ship — judgment about eval tradeoffs.
8. Someone is accountable when a shipped model causes a harmful, biased, or costly decision.
9. Production monitoring — catching data drift and silent degradation — requires ongoing human-owned judgment.
10. Deciding which metric or tradeoff (latency vs. accuracy vs. cost) actually matters to the business is a judgment call, not a technical one.

Invalid axioms

1. Writing training/inference code and pipeline boilerplate requires scarce specialized engineering time. LLMs generate working PyTorch/JAX code, data loaders, and standard pipeline scaffolding on demand — this was the bulk of junior ML engineering headcount. Habit-trap: teams still hire and structure ladders around "can write clean training code" as if it were the differentiator, when it's now a commodity output any competent engineer can get from a model in minutes.
2. Reproducing state-of-the-art results requires scarce access to papers and the expertise to translate them into working code. LLMs read a paper and draft a working implementation attempt fast, collapsing the translation gap that used to take a strong engineer days. Habit-trap: orgs still budget "paper-to-prototype" as a multi-week specialist task instead of a same-day draft-plus-verify cycle.
3. Feature engineering and architecture choices require scarce human trial-and-error intuition. Agentic tooling can now run large sweeps — architectures, hyperparameters, feature sets — at a speed and volume no human loop matched, because search and pattern-matching against prior published results is exactly what the model is good at. Habit-trap: teams still gate experiment design on a senior engineer's manual intuition pass before anything runs, when breadth-first automated search now finds the same ideas faster and cheaper.
4. Compute is scarce and expensive, so experiments must be planned carefully before running. This is flipping, not flipped — compute is still real money, but the planning bottleneck (writing the experiment code, wiring the sweep, analyzing results) is what got cheap, not the GPUs themselves. Habit-trap: teams still route experiment design through slow manual review cycles calibrated to an era when writing the experiment was the expensive part, not just running it.

Unchanged axioms

1. A human must decide whether a model is good enough to ship. A model can generate an eval report; it cannot own the tradeoff between a 2% accuracy gain and a fairness regression, or decide that "good enough" depends on context an LLM wasn't given. Judgment under this kind of ambiguity is exactly where models still fail confidently.
2. Someone is accountable when a shipped model causes a harmful, biased, or costly decision. No amount of LLM assistance changes who gets named in the postmortem or the regulatory filing. Accountability doesn't scale with token throughput — it stays pinned to a person or org.
3. Deep debugging of model internals requires engineers who understand the math and systems together. LLMs are confidently wrong about subtle numerical bugs, silent tensor shape mismatches, and distributed-training race conditions — the failure modes that don't pattern-match to anything in training data because they're specific to this exact stack and this exact run. Ground-truth verification of "why did the loss curve do that" still needs someone who can reason from first principles, not retrieve a similar-looking Stack Overflow answer.
4. Production monitoring for drift and silent degradation requires ongoing human-owned judgment. Detecting that a model's real-world behavior has quietly diverged from its training distribution is a live, contextual, high-stakes call — the exact kind of novel-ambiguity judgment that stays scarce regardless of how good the model's own self-description is.
5. Deciding which metric or tradeoff matters to the business is a judgment call, not a technical one. An LLM can list the tradeoffs; it cannot decide that this product should optimize for recall over precision because of a specific customer commitment. That's taste and goal-setting, not synthesis.

New axioms

1. When any engineer can generate a plausible model, pipeline, or paper reproduction in minutes, who verifies it's actually correct before it reaches production? Volume of plausible-looking ML code has exploded faster than teams' capacity to review it for the subtle bugs that don't show up until a model is live — this is a review bottleneck that didn't exist when writing the code itself was the rate-limiter.
2. When experiment generation and hyperparameter search run at machine speed, how does a team avoid drowning in results it can't triage? Abundant automated experimentation produces more candidate models and more eval reports than any team can meaningfully interpret, shifting the bottleneck from "can we run this experiment" to "can we tell which of these hundred runs actually means something."
3. When an LLM can write a benchmark-beating model from a paper in an afternoon, how does a team tell genuine capability from overfitting to a benchmark the model has effectively memorized? Pattern-matching against everything ever published makes it cheap to hit known benchmarks and easy to mistake that for real generalization — verification against novel, out-of-distribution reality becomes the actual test, and nobody has a fast version of that yet.
4. As agentic coding tools take on more of the ML pipeline autonomously, who is responsible for auditing a pipeline no single human fully wrote or read end to end? Accountability historically attached to the engineer who wrote the code; when code is agent-authored and stitched together at speed, ownership of the artifact is not automatically decided.

Where it breaks

"Writing training code and reproducing papers is the scarce skill" (invalid) collides head-on with "someone must verify the model is actually correct before shipping" (new): teams are using freed-up engineering time to generate more models and more experiments faster, not to build the verification capacity that abundance now demands — the review bottleneck grows exactly as fast as the code-generation bottleneck shrinks, and headcount hasn't moved from one to the other.

A second collision: "compute is the expensive constraint, so experiments are planned carefully" (invalid, since planning is now cheap) runs into "abundant experiment volume creates a triage problem no one owns" (new) — teams that removed the planning friction without adding a way to interpret a hundred automated runs are burning compute faster while understanding less per dollar spent, not more.

ai-axioms.vercel.app/axioms/engineering/ml-engineering-ai-axioms