The Year I Stopped Blaming the Model

A year ago, I asked for a repository layer. One layer, one pattern, applied across our domain objects. What came back were twenty files that technically worked and read like twenty different engineers had written them without ever talking to each other — a different assumption about structure in every one, nothing holding the pattern steady from one domain object to the next.

I rewrote the prompt. I got a different kind of inconsistency back — still technically valid, still divergent in a new way. I rewrote it again. Same outcome, different flavor of wrong. There was no magic phrasing sitting a few tries away. The problem was never really about phrasing.

This was everybody's year, even if we didn't compare notes on it. Ask for a component and get five different assumptions about what "the same" means. Hand it five steps and get three done correctly, a fourth invented, and a fifth skipped without a note.

It wasn't just the output that was inconsistent — the agents themselves were, too. The early ones we built ran on whatever model was cheapest, because it was easy to justify cheap when you weren't sure the whole idea would work yet. They didn't feel like collaborators. They felt wooden — like a vending machine that only dispensed if you found the exact right combination of words. Trigger phrases. Magic incantations that happened to work on Tuesday and stopped working on Thursday. We were genuinely new to this, and there's a particular kind of ground you have to make up when you don't yet know what you don't know.

The natural conclusion, reached out loud or quietly by nearly everyone building with this stuff, was: it doesn't really follow instructions. It's not that good at this. It can't really be trusted with production code.

That conclusion wasn't wrong. It was aimed at the wrong layer.

We were grading the model against a standard it was never given. "Consistent" was a word in a prompt, not a structure in the system. Nothing told it what "the same pattern" meant beyond whatever sentence happened to be in front of it that day — no shared template it had to match, no check that would catch it if it drifted, no shape it had to conform to that existed anywhere outside the words we typed. Of course twenty files looked like twenty different answers to the same question. In a very real sense, they were twenty different questions — one per prompt, one per run, one per however-I-happened-to-phrase-it that time.

The models are better now, across the board. That's real, and picking one because it was actually good at the job — not just the cheapest one that could technically do it — made more of a difference than I expected. But none of that, by itself, creates codified determinism. A better model still has no shape to conform to unless you build one; it just writes the divergence with better grammar.

What actually turned this around had a name attached to it. A colleague, Scott Blevins, looked at what we'd built — the wooden agents, the magic words, the whole improvised mess — and suggested we try a different framework: Agno. That's the moment this stopped being trial and error and started being systematic. Instructions had a real structure. Tools had a shape the agent could actually reason about, instead of a name it had to guess the right incantation for.

Around the same time, we started writing our tools with real inline documentation — not comments for us, comments the agent itself could read to understand what a function did and how to use it correctly. It changed what felt possible. Suddenly it felt like I could point an agent at almost any problem and it would find its way to the right tool for the job.

It could. It did. That's a different story, and I'll get to it — because capable and controlled turned out not to be the same thing at all. I stopped blaming the model around here, not because it started behaving, but because I understood something more useful: if nothing is actually keeping an agent in check, it can and may do anything. Capability was never the constraint I thought it was. What was missing was something to hold that capability inside a shape — the same kind of shape that would have kept twenty repository files looking like one pattern instead of twenty.

That's the thread I want to pull on all week: not "will it do the right thing if I ask nicely," but "what has to be true in the system so it can't do the wrong thing." That's not a prompting question. It's an engineering question, and it has a completely different kind of answer: structure. A coding harness — guardrails that block instead of merely suggest, conventions the agent can check itself against instead of guess at, verification that runs after the work is claimed done, because a claim and a fact are not the same thing until something has actually checked.

Picture that same repository-layer request run through a harness instead of a raw prompt. There's a template or a schema the new file has to match — not a sentence describing what "matching" should look like, but a shape it structurally has to conform to. There's a check that runs before the file is accepted, not a hope that it turned out fine. If domain object twenty-one shows up needing a repository, it doesn't get a fresh guess at what "consistent" means that day — it gets built against the same shape as the other twenty, because the shape is the thing being enforced, not the mood of a prompt.

This week, I want to walk through the specific lessons that got me from one side of that line to the other — not prompt-engineering tricks, but the structural ones. The stuff that actually moved the needle when I was building agent workflows, not just asking a chatbot for help.