When a self-driving car crashes, nobody sues the AI. They sue the company. They sue the safety driver. They sue whoever decided the system was ready for the road. The AI doesn't own a bank account. It doesn't hold a license. It can't be held in contempt, can't be fired, can't lose anything. It was never a party capable of being liable in the first place — so of course it isn't.
That's not a temporary gap that next year's model closes. It's not a maturity problem, waiting on more parameters or better training. Liability isn't a capability. You don't get better at it the way you get better at writing code or classifying an image. It's a legal and moral status — the capacity to actually be held responsible for an outcome — and it belongs to people, and to the organizations they run. Not to the tool they used to get there.
This is easy to nod along with in the self-driving-car version, because the stakes are dramatic and the example is familiar. It gets harder to hold onto once you bring it back down to something as ordinary as code review.
Code review is where this actually gets missed. A pull request can pass every automated gate you've built — the tests are green, the types check, the linter's silent, the diff matches the shape it's supposed to match — and it can still be wrong. Not "wrong" in the sense of violating a rule nobody wrote down yet. Wrong in a way none of your mechanisms were ever built to catch, because the mistake isn't a rule violation. It's a judgment call that happened to go badly. Every check you own answers "does this match what we already decided correctness looks like." None of them answer "was this actually the right thing to ship." That second question is what a human reviewer is there for — not to re-run the linter with more attention, but to be the person who's actually answerable if the answer turns out to be no.
Raise the stakes further and it stops being subtle at all. Take something like classifying food allergens — is this product safe for someone with a nut allergy. A model can get very good at this. It might even get it right nearly every time. "Nearly every time" is still not the same thing as "liable." No matter how reliable the classification gets, someone still has to be the last word before that label goes on that product — a person who can be held responsible if it's wrong, because being wrong here doesn't cost a failed test, it costs someone's health. The model being right most of the time was never the question. The question was always who's allowed to be the one who's sure.
Once you see it this way, there are really two different categories of decision hiding under the single phrase "needs human review," and they don't work the same way at all.
The first category is liability. Anything where being wrong costs a real person something real — their safety, their money, their trust, their freedom — needs a human who can actually be held responsible for the call. This isn't a knock on the technology, and it isn't a line that moves as the models improve. It's a category error to expect it to move, the same way it would be a category error to expect a car to eventually become a defendant. Whatever this generation of AI becomes, and whatever the next one after it becomes, none of it changes who's capable of being sued, fired, or held to account. That was never a technical question.
The second category is judgment — design opinion, architectural taste, how you actually want a piece of code written when there's more than one defensible way to write it. This one's different, and I'll be honest about where I land on it: current AI mostly isn't there yet, and I genuinely don't know if this class of AI — the large-language-model kind we're all building on right now — ever gets there. Taste isn't just pattern-matching against what's been seen before. It's a stance. It's a set of trade-offs somebody actually cares enough to defend, accumulated the hard way, usually by being wrong about them first. Maybe a future system closes that gap. Maybe it doesn't. I'm not going to pretend I know which, and I'd trust the answer less from anyone who claims they do.
Both categories end up at the same practical answer, even though they get there differently: a human stays in the loop. Not because the mechanism failed — the argument all week has been that you should build all the mechanism you possibly can, and I still believe that. But mechanisms only ever answer "is this correct." They were never going to answer "is this right," and for the decisions that actually carry weight, they were never going to answer "who's responsible if it's not." That's not a hole in your tooling waiting to be patched by a better model. It's the one part of the job that was never automatable in the first place, because it was never really about capability. It was about who's left holding it when something goes wrong.
The agent isn't liable. You are. That's not a limitation to design around — it's the actual reason the review exists.
Stay Updated
Get notified when I publish new articles on development, architecture, and tech.