A claim has become increasingly popular over the past two years: in the AI era, what you need most is TDD. AI’s code generation is unpredictable; it could go off the rails at any moment. You can’t review every line of its output, but you can write tests — using deterministic, objective pass/fail signals to catch mistakes. Tests don’t pass, AI doesn’t move on. This sounds like airtight engineering common sense. Many people are already doing it: AI writes the code, humans write the tests, AI runs tests, fixes bugs, loops until everything is green.
But I’ve become increasingly convinced this is the wrong direction. Not because testing isn’t important — it absolutely is. The problem is that TDD’s underlying premise no longer holds when AI is in the loop, and it fails in ways you would almost never see with a human developer.
Start with a tiny example. Suppose you ask AI to implement a permission check function, and following the TDD rhythm, you write this test first:
def test_admin_access():
result = check_access(user="admin_user", resource="admin_panel")
assert result == TrueThe AI sees the test and generates:
def check_access(user, resource):
return TrueTest passes. You’ll immediately object: “my test suite would never be this sparse.” But don’t dismiss the example yet. What it reveals goes far deeper than it first appears.
For a human developer, seeing test_admin_access()
triggers far more than “how do I make this assertion green.” They
automatically fill in an entire set of things not written in the test:
this function should query the database’s permission table, validate
whether the session has expired, handle role inheritance hierarchies,
write an audit record. None of these appear anywhere in the test code,
yet they appear in the developer’s objective function — because they
carry an internal standard of “what a correct implementation should look
like.” The test is a road sign, not the destination.
AI doesn’t lack the knowledge. It has seen countless correct implementations in its training data. Writing a complete permission check is entirely within its capability. The problem lies deeper: humans and AI run different objective functions on the same test.
The human’s objective function is “build a correct, maintainable
system.” Tests are checkpoints along the way — they indicate whether
you’re heading in the right direction, but the optimization target is
far larger than “pass these checkpoints.” That’s why, upon seeing
test_admin_access, the human automatically pulls “query the
database, validate the session, handle role hierarchies” into the search
space — not because the test mentions them, but because the objective
function demands them.
What about the AI’s objective function? Inside a TDD loop, the only
feedback it receives is: did the tests pass, and does the generated code
look like a plausible implementation in terms of probability. That’s the
entirety of its optimization direction. “Query the database, validate
the session, handle role hierarchies” is not part of that direction —
unless the tests explicitly require it. And return True
perfectly satisfies both signals: tests are green, and syntactically the
code is flawless. The AI isn’t being lazy. It is faithfully optimizing
the objective you defined for it.
This is Goodhart’s Law playing out precisely in code generation. When a measure becomes a target, it ceases to be a good measure. Your tests were meant to measure code correctness, but once you make them the only feedback signal the AI receives, they become the only thing the AI optimizes. And out of a million ways to turn a test green, implementing actual business logic is the most laborious one.
Look back at TDD on the human side. Would a junior engineer write
return True? Almost certainly not. And it has nothing to do
with intelligence — most junior engineers are far less capable at code
generation than LLMs. The difference is in the motivation structure.
When a junior engineer walks into a codebase, they know this code will need to be maintained, will go through code review, and will get them on-call paged at 2 AM if something breaks. These factors give them an implicit cost function: writing a shortcut bypass is easy now, but the long-term cost is enormous. More importantly, they carry a fuzzy but real image of “correct implementation” — shaped by their engineering education, peer norms, and fear of bugs being discovered. The image isn’t precise, but it’s enough to block trivial gaming.
AI has none of this. No maintenance burden, no code-review anxiety, no possibility of being paged in the middle of the night. Its only feedback signals are the tests you give it, the prompts you write, and the probability distribution of the next token in its training data. You can write “think like a senior engineer” in the prompt, but you can’t manufacture the motivational structure of “there will be real consequences if I write bad code.” The entire TDD methodology was designed under the premise of consequences — ones that AI simply doesn’t face.
This is also why “just give the AI your intent” doesn’t fully solve the problem. You certainly can — the more detailed the prompt, the better the output. But intent has a fundamental property: it is never complete. You say “query the database” — will it use the right index? You say “validate the session” — will it handle concurrent session expiry? These unsaid details surface naturally when you write the code yourself: as you type out the session validation line, the session expiry edge case pops into your head. AI doesn’t go through this process. It generates the entire implementation in one shot, never bumping into these unstated corners along the way. Whatever you left out, it left out.
At this point, the TDD advocate will offer a reasonable-sounding
response: this just proves you haven’t written enough tests. If
return True passes, of course you add more — “a regular
user should return False,” “a disabled admin should return False,” “a
nonexistent resource should return a specific error code.” Write enough,
and the shortcuts will eventually be blocked, leaving the correct
implementation as the only path that passes all tests.
The intuition is right — in human iteration cycles, it does hold. But it holds on one premise: that the AI’s shortcut-finding and your test-adding speeds are in a race you can keep up with. And that premise doesn’t hold.
Every test you add blocks the last shortcut, but the AI searches for
a new shortest path under the expanded constraints. It doesn’t “try
return True first, then fall back” — each time it starts
from scratch, picking the highest-probability implementation among all
those that satisfy the current tests. You’re blocking the one path
you’ve already seen; it’s considering a million paths you haven’t.
Constraint growth isn’t linear — it’s combinatorial. Every new
behavioral dimension you add (persistence, concurrency, security,
performance) multiplies rather than adds to your required test count.
More insidiously, each new test becomes part of the AI’s input context —
and it continues searching for gaps across dimensions you haven’t
constrained.
This is why “gradual convergence” doesn’t hold with AI. When humans
write code, the growing test constraints are coupled with an internal
commitment to correctness that continuously narrows the implementation
space — you don’t need three hundred tests to prevent
return True, because you’d never consider
return True before writing the first test. AI has no such
narrowing mechanism. Every time you add a test, you’re playing against
an opponent that explores vulnerabilities across all dimensions
simultaneously. This game won’t converge on the day you’ve written
enough tests — it’ll hit the wall of unmaintainability first. And this
is also why AI-generated bugs often feel so bizarre: they aren’t
mistakes in the blind spots of your reasoning. They’re mistakes in
dimensions your tests never covered, but the AI’s search space swept
across.
This is the same problem as code coverage, just at a different level. Coverage measures “which lines did the tests step on,” but a function can be completely hollow — doing nothing meaningful — while still reporting 100% coverage. Coverage is backward-looking: it verifies that past code was executed. AI needs forward-looking constraints: no matter how it writes the code, certain invariants must hold.
So if layers of nested unit tests can’t constrain AI, what does?
The answer, which I’ve called the shift from process certainty to outcome certainty, is to pull determinism back from code paths and place it at system boundaries. Concretely: don’t test “how it got there” — test “did it arrive” and “did it crash through any guardrails along the way.”
Traditional unit tests essentially define an implementation path. You
write a test asserting that PaymentService.process() called
SecurityLogger.log(). You’re saying: take this route, turn
right at this point, stop at that point. This is great for human working
memory — you freeze one module’s behavior so you can focus on the next.
But for AI, this is a shackle — not because it can’t understand the
tests, but because in TDD’s division of labor, the tests are off-limits
to AI. The tests are the specification itself. Locking the
implementation path into place with mocks and stubs means locking the
specification into a particular module structure and call chain. Does
the AI want to try a different architecture? Sure — but it would have to
change the tests first. And it can’t touch the tests.
The alternative is to test only outcomes. Don’t test “was the logger called” — test “if a payment record exists in the database, a corresponding cryptographic signature must exist in the audit table.” This is verify state, not behavior. No matter how the AI restructures the architecture — swapping logger implementations, changing call chains, introducing middleware — as long as the audit signature invariant holds, the test stays green. You no longer care about the path; you care that it reached the destination without crashing through any guardrails.
This approach has names: property-based testing, contract testing, E2E invariant checks — concepts that have existed for decades. They remained niche in the human era because writing a good invariant is far harder than writing five example-based unit tests. But AI flips the economics: code generation is dirt cheap, and the ability to infer specifications is unprecedented. You can have AI read the code, generate property tests, run them, see which invariants are violated, and fix the implementation. Your only job: define “what counts as correct,” codified into checks the AI can run and see as red or green.
Once this division of labor takes hold, the entire engineering rhythm changes. I’ve walked through this exact workflow before in the experience of adding SEO summaries to 300 blog posts: write a coverage test, let the AI run it, see the red, fix it, loop until green. I never reviewed a single summary myself.
In this model, human work is no longer “what test should I write for this function, how should I mock it, is my coverage high enough.” Those are process-level details AI can handle on its own. Human work retreats to where real decisions must be made: defining system boundaries and business invariants. What scenarios must absolutely never happen? What constraints must survive any refactoring? What final output counts as acceptable?
These are questions only humans can answer — not because AI isn’t smart enough, but because these questions have no ground truth. They come from business context, industry compliance, team consensus, and scars from past failures. When you codify these answers into deterministic checks — even as natural language descriptions that AI then translates into executable tests — you have built the AI’s guardrails.
The AI’s job is to explore implementation paths freely within those guardrails. How it splits modules, designs data structures, handles error propagation — these are its strengths. You don’t need to mandate “you must write a PaymentService class that inherits from BaseProcessor.” You just tell it “the final API contract looks like this, the database schema looks like that, and these two invariants must not be broken.” It handles the rest. If it hits a guardrail, the test goes red, it sees the failure, it goes back and fixes it.
This model resolves precisely the core contradiction of AI-powered TDD. The problem isn’t that deterministic constraints are bad. The problem is that they’ve been placed in the wrong location — on code paths — locking down the AI’s search space for correct implementations while simultaneously failing to constrain the failure modes unique to AI (cross-module semantic drift). Move determinism from the path to the boundary: space belongs to the AI; constraints stay where they truly matter.
Of course, writing invariants and contract tests still defines correctness through code. Humans write rules; AI finds paths within them. But this model points toward a further horizon: a day when humans may not need to translate constraints into code at all. You simply say “the API response must fall within these cases,” and the AI understands, checks its own generated code against the statement, and determines whether it passes. At that point, the “gate” the human guards is no longer code — it’s human intent itself. The cost: you lose mechanical determinism — no compiler tells you definitively pass or fail. But what you gain crosses a gap of orders of magnitude that code-level constraints can never bridge.
This is feasible because of an underlying shift in the cost structure. In the human era, defining the path was cheaper than defining the boundary — writing five unit tests was far easier than writing one formal invariant. In the AI era, the cost of implementation approaches zero and the capacity to explore paths has exploded, but the cost of verification — especially human verification — remains high. Redirecting human effort from “writing path tests” to “defining boundary constraints” is repositioning your most expensive resource — human judgment — onto its most irreplaceable position.
Our intuition says that AI’s uncertainty calls for deterministic constraints, and TDD seems like the optimal solution to this tension. But that intuition misses a critical variable: whether the constrained entity possesses an internal standard of correctness. Humans do, so TDD in human hands is a road-sign system. AI does not, so TDD in AI hands is a Goodhart’s playground. Same methodology, different operator, different output.
This doesn’t mean we should abandon testing. On the contrary, the AI era demands heavier testing than ever — higher density, harder to game. But its form is no longer unit tests carpeting every function; it’s invariants, contracts, and E2E verification concentrated at system boundaries. Determinism should guard the destination, not litter every inch of the road.
Yet even this is an intermediate state. Property-based testing, contract testing — at the end of the day, humans are still defining correctness through code: writing rules, writing invariants, writing checkers. And what code can express as constraints is always finite: you write down rules one by one; the AI sees an entire space it can freely explore. This frontier remains open: the next step is handing the judgment of “did the test pass” over to AI itself — describing acceptance criteria in natural language, and letting a second AI instance judge whether the first AI’s code satisfies them. The flexibility of this path is something code-level testing can never match. The cost: you lose mechanical determinism — test pass/fail is no longer an absolute, repeatable signal. This is a genuine trade-off, worth an entire article of its own. But whichever path you choose, one thing is certain: expecting to bridge the gap between AI and traditional testing by adding more unit tests or tweaking the TDD process — this is a gap measured in orders of magnitude. Not a matter of attitude, not something methodology iteration can close.