Three Things Jason Gorman Got Right About AI, and Three I See Differently

Software Engineering Radio is the one podcast I never skip. It has been running since 2006 with the same stubborn format, a working engineer interviewing another working engineer, and nobody is trying to sell me an observability platform halfway through. Episode 732, Jason Gorman on The Effective Use of AI for Software Development, came out at the start of August and I have now listened to it twice: once walking to the office, once while pretending to do laundry.

Gorman runs Codemanship and has spent roughly two decades teaching TDD, refactoring and continuous delivery. That background matters, because his take on AI is not the usual split between "it will replace all of us" and "it is just fancy autocomplete." His claim is narrower and much more useful: the practices we already knew were good become more important when a model writes the code, not less. Giovanni Asproni hosts, and he pushes back in the right places.

So, a summary of what the episode says, and then the three points where I landed somewhere different.

What the episode actually says

1. Modularity is a context-window strategy

The strongest idea in the hour is the line Gorman draws between modular design and model performance. LLMs degrade when you drown them in a large, unstructured codebase. They lose the thread, they invent a helper that does not exist, they "fix" a bug by quietly rewriting a function three files away. The answer is not a cleverer prompt. The answer is a codebase where the relevant unit of work is small enough to hand over whole.

He has packaged this into the CRESS principles for context engineering: a context should be Current, Refutable, Empirical, Small and Specific. Small is the one that stuck with me. His claim, from his own closed-loop experiments, is that if you work in small tested steps you can usually keep a context down to a few hundred tokens, comfortably inside the range where models are still reliable.

This maps onto something I hit constantly. Handing an agent one isolated, well-named function with a clear contract produces dramatically better output than pasting in a 900-line pipeline job and hoping. Which is, when you squint, exactly what Parnas wrote in 1972 about decomposing systems into modules. We just have a new reason to care.

2. Spec-driven development as the guardrail

The second theme is what happens to our job. If the model writes the implementation, the human moves up a level, to defining constraints: specifications precise enough to be unambiguous, and automated tests that verify behaviour instead of trusting it.

This is where TDD stops being a style preference and becomes infrastructure. Tests are the thing that catches code which reads beautifully, compiles, passes a review by eye, and silently drops the last row of every batch. Gorman's Refutable and Empirical principles are precisely this: the output has to pass through a gate that can tell you, with confidence, when intent was not satisfied. Without that gate you are just reading generated code and nodding.

The tooling has caught up fast. GitHub's Spec Kit is basically this workflow productised, and the Ralph loop is the same instinct taken to its logical extreme: a fresh context every iteration, one task at a time, and a spec acting as the lookup table.

3. Cognitive debt

The part I keep coming back to is the psychological one. Gorman talks about cognitive debt, offloading so much of your thinking to the model that you quietly lose the structural understanding of your own system.

There is uncomfortable evidence for this now. The MIT Media Lab EEG study on LLM-assisted essay writing found weaker neural connectivity, weaker recall and a weaker sense of ownership in the group that used an assistant. And METR's randomised trial of experienced open-source developers found they were 19% slower with AI tools while believing they had been 20% faster. That gap between felt productivity and measured productivity is the scariest number in this field right now.

His practical rule follows from it: the code has to stay readable by humans, whoever typed it. If you cannot debug the model's output quickly, the time you saved generating it gets refunded with interest the first time production goes sideways at 2am.

Where I see it differently

Everything above is right for the context Gorman is describing: a system that has to run in production for years, maintained by a team. Where I end up somewhere else is on the boundaries of that context.

1. The small-context mandate optimises for the wrong strength

Keeping contexts small is good hygiene, and the research does back the degradation claim. Chroma's context rot work tested 18 frontier models and found every single one gets worse as input grows, often well before the advertised limit. Lost in the Middle showed the same shape years earlier.

But I think "keep the context small" is the wrong conclusion to draw from that evidence. What degrades with length is precision recall. That is not the same as saying broad context is useless, it means broad context is bad at needle-in-a-haystack and good at something else entirely.

And that something else is the thing humans are worst at. An agent that can see the whole repository can notice that three services implement the same retry logic with three different backoff bugs. It can spot that a column renamed in an ingestion job is still referenced by a dashboard query living in another repo. It can carry a repo-wide refactor with a consistency no human sustains past file forty. A strictly module-by-module workflow structurally cannot find those things, because the problem does not live inside any module.

So my version of the rule is: small contexts for execution, broad contexts for architecture. Narrow window when I want a correct diff. Wide window when I want to know what is wrong with the system as a whole, and then I verify what it tells me, because at that width it is generating hypotheses, not facts.

2. Strict TDD taxes the part of AI that is genuinely new

Gorman is right about production. For anything that ships to 70+ countries, specs and tests are not negotiable, and I would not want to work any other way.

But applying full TDD ceremony to every single interaction with a model ignores what these tools actually unlocked, which is throwaway architecture. I can ask an agent for a complete, running prototype of a feature, no tests, no spec, deliberately disposable, purely to find out whether an architectural approach survives contact with reality. That used to cost two days, so instead I would argue about it in a design doc. Now it costs twenty minutes, so I can build three versions and look at them.

Writing the spec first assumes you already know what you are specifying. In the exploratory phase you usually don't. That is the entire point of the phase.

The discipline I actually need is not "always TDD." It is knowing which mode I am in and being honest about it, and never letting the reckless-brainstormer output graduate into the repository without being rewritten under the disciplined rules. The failure mode is not prototyping without tests. It is forgetting which one you were doing.

3. "Humans must read all of it" is true now, and probably temporary

The claim that AI-written code must stay fully readable to humans is correct today, and I follow it today. I am much less sure it is a permanent law.

We do not read the assembly our compilers emit. We stopped around the point where the compiler became more trustworthy than a human doing the same transformation by hand, and the source language became the thing we reason about instead. CompCert and seL4 pushed that further: artifacts we trust because of a proof, not because anyone read every line.

I can see the same trade arriving for narrow, hot, hyper-optimised functions. If the specification is precise, the property-based tests are exhaustive, the interface is small and the thing is genuinely isolated, then human legibility of the internals becomes a nice-to-have rather than a requirement. The architecture stays human-owned. The boundaries stay human-owned. The middle of a 200-line numerical kernel might not need to be.

That condition is enormous, though, and here Gorman would push back and be half right: almost nothing in a real codebase is that isolated. Today the honest answer is that you should read it. I just do not think "you should read it" is a fact about software. It is a fact about how good our verification currently is, and that is a moving number.

So where does that leave me

Rereading my own three points, they are all really the same point. Gorman is describing the discipline of maintaining a long-lived system, and I keep reaching for the exceptions at the edges, the exploratory phase, the whole-repo view, the isolated kernel. Those exceptions are real. They are also, if I am honest, a much smaller slice of my week than the maintenance is.

The 2025 DORA report put it in a sentence I keep coming back to: AI is an amplifier. Strong teams get stronger, struggling teams get their existing problems delivered faster. Nothing in the episode contradicts that, and neither does anything I have written here. The specs, the tests, the modules, that is the amplifier being pointed at the right signal.

Go listen to it. It is a good hour.