No. 42 / 339
Is manual test-case writing dead now that AI can generate test cases from a user story?
The shift
Producing a structurally complete first draft of test cases — happy path, boundary, negative, edge — from a user story goes from scarce, hand-authored work to abundant: instant, near-free, and generatable at any volume. That's pattern-matching against "what test coverage looks like for this shape of requirement," a core LLM strength, not judgment about whether the software actually works.
The axioms
- Translating a user story into a structured set of test cases requires trained QA judgment, and that judgment is scarce and billed by the hour.
- Comprehensive coverage — boundary values, negative paths, edge cases — comes from experience and requires deliberate, effortful enumeration.
- Writing test cases in the house format (Gherkin, step tables, whatever the test-management tool wants) is a specialized authoring skill.
- Acceptance criteria are incomplete by nature; a human tester fills the gap with real knowledge of how the system actually behaves, not just what the ticket says.
- Someone has to decide whether a test case is actually worth having, and be answerable when a shipped bug slips through untested.
- Finding the bug nobody thought to write a case for depends on human curiosity and lateral thinking, not enumeration from a spec.
- Regression suites drift out of sync with the product, and closing that drift is bottlenecked by tester time to notice and update.
Invalid axioms
- Translating a user story into a structured set of test cases requires trained QA judgment. The scarce input here was authoring time, not insight — turning "as a user I want X" into a table of happy-path/negative/boundary cases is a format-and-pattern task, exactly what a model does well and fast. Habit-trap: teams still staff test-case authoring as dedicated QA hours per story, and price sprints assuming that drafting is the expensive step.
- Comprehensive coverage requires deliberate, effortful enumeration. A model asked for edge cases will reliably list more boundary conditions, input permutations, and negative paths per minute than a person drafting by hand — enumeration is a volume problem, and volume is now free. Habit-trap: "coverage" is still used as a proxy for tester effort invested, when it's now a cheap byproduct of asking twice.
- Writing test cases in the house format is a specialized authoring skill. Formatting to Gherkin, step tables, or a specific test-management schema is exactly the kind of style-translation LLMs do without friction. Habit-trap: onboarding and style guides still treat "write it in our format" as a trained skill worth protecting, rather than a one-line prompt instruction.
Unchanged axioms
- Someone has to decide whether a test case is actually worth having, and be answerable when a shipped bug slips through untested. A model can generate a hundred plausible cases; it has no stake in the release and no liability when the real bug was the one nobody asked it to check. Deciding which 20% of generated cases matter for this release, and owning that call, stays human.
- Acceptance criteria are incomplete, and someone must fill the gap with real knowledge of how the system behaves. The AI only sees the story text — it doesn't know the legacy quirk, the flaky third-party integration, or the fact that the last three incidents came from a specific race condition. That knowledge lives in people who've operated the system, and it's what separates a test case that catches a real failure from one that only looks thorough.
- Finding the bug nobody thought to write a case for depends on human curiosity and lateral thinking. Exploratory testing isn't enumeration from a known spec — it's poking at the product with intuition about where things break. LLM-generated cases are bounded by "what does a test case for this pattern usually look like," which is a different activity from noticing something's off during a session.
New axioms
- Test suites can now grow far faster than anyone can review them for redundancy or blind spots. When generating 50 more cases costs nothing, the open problem becomes curation: pruning near-duplicate cases, catching false confidence from a suite that's wide but shallow, and preventing "test count" from being mistaken for "coverage quality."
- Confidently-plausible-but-wrong test cases now enter suites at volume. A generated case that tests the wrong assertion, or asserts against a misunderstood spec, looks identical in format to a good one — and at scale, someone has to catch the ones that pass green while testing nothing meaningful. Verifying the verifiers is now the bottleneck, not writing them.
- The tester's job shifts from authoring to reviewing AI output under time pressure, and that shift hasn't been resourced. If test-case drafting collapses from days to minutes, teams either bank the time saved for deeper exploratory work and review, or they silently expect the same headcount to now also own AI-output verification on top of everything else it did before.
Where it breaks
Teams still budget QA hours as if test-case authoring is the expensive step (invalid), while the actual new bottleneck is reviewing AI-generated suites for redundancy and false coverage at volume (new) — sprints get "more test cases" without anyone owning whether those cases are worth trusting. Separately: acceptance criteria are treated as sufficient input for AI-generated coverage (invalid, since teams now lean on the model to fill gaps), while the real gaps in acceptance criteria are exactly what only a human with system knowledge can see (still holds) — the story-to-test-case pipeline gets faster without anyone checking whether the story itself said enough.
Related axioms
Engineering
What changes for data engineering with AI?
Engineering
What changes for DevOps with AI?
Engineering
What changes for hardware engineering with AI?
Engineering
What changes for ML engineering with AI?
Engineering
What changes for QA and testing with AI?
Engineering
What changes for software engineering with AI?
Other axioms
Healthcare
Does public health surveillance still require large epidemiology teams when AI can detect outbreak signals from data in real time?
Management
What changes for management consulting with AI?
Finance
What's the point of the CPA credential when the software already knows GAAP?
Finance
What changes for insurance with AI?
Legal
Is the paralegal role dead, or does it just move upstream into AI-output verification?
Architecture
Should a licensed architect's stamp still mean the same thing when the underlying model was AI-authored?