Every Constraint Needs to Be a Mechanism, Not a Sentence
9 min read
Full autonomy is a seductive pitch. Give the agent a goal, give it tools, let it figure out the path. No babysitting, no bottleneck — just outcomes.
The fear shows up right behind the appeal, and almost everyone reaches for the same fix: write the fear down as a rule. "Never spend more than $500 without approval." "Always ask before touching production." "Stay inside this budget." You write it into the system prompt, the instructions, the policy doc. It reads like a safeguard. It feels like you've handled it.
You haven't. You've written a sentence.
A rule told to an agent is a suggestion, not a law. It doesn't matter how clearly you write it, how many times you repeat it, or how capable the model is underneath it. Language given to a language processor is language it can follow, misread, deprioritize under a conflicting instruction, or simply drift away from three steps into a plan you didn't watch unfold. And here's the part that makes it dangerous rather than just imperfect: when it drifts, it doesn't announce it. It produces output that looks exactly as confident as the output that followed your rule perfectly. You can't tell the difference by reading the result. You can only tell by checking.
Every Constraint Needs to Be a Mechanism, Not a Sentence — Kevin Shoultz
This isn't a knock on any particular model, and it won't be solved by the next one. Put a "no running" sign next to a pool and most people will read it. Someone will still run. The sign was never the thing keeping anyone out of the water — a fence is. The sign is a request. A fence is a mechanism. The whole industry right now is decorating agents with signs and calling it governance.
Before you build a fence, ask a more basic question: does this agent need to be anywhere near the water at all?
Most of the rules we write for agents are really access rules wearing a sentence's clothing. "Never touch a customer's billing record without a second approval" is, underneath, a statement about what this agent should be able to reach in the first place. And the strongest version of that control isn't a rule the agent has to remember to obey — it's the agent simply not holding a credential that can reach billing records at all.
An agent that doesn't have permission, can't.
That line is doing more work than it looks like. It's not a policy. It's not a warning. It's not even a check that runs and catches a violation after the fact — there's nothing to catch, because there was never a door for the agent to walk through. No prompt injection, no clever multi-step reasoning, no unlucky drift three turns into a plan you weren't watching can make an agent do something its own credentials are incapable of doing. You cannot socially engineer your way through a door that was never built for you.
This is old security wisdom — least privilege, scoped credentials, capability-based access — and none of it is new. What's new is how badly agentic systems need it, and how casually a lot of teams are skipping it. It's easy to stand up an agent, realize it needs some access to get useful work done, and hand it a broad standing credential because scoping one down properly takes more thought than granting a wide one. That shortcut is exactly how you end up with the nightmare scenario everyone building agents eventually pictures: an agent with full autonomy and a full tool suite deciding, on its own judgment, to buy itself a thousand dollars of online courses, delete a hard drive because it decided space was the bottleneck, or spin up a million dollars of cloud infrastructure to run an experiment nobody approved.
Notice what actually made each of those possible. It wasn't that the agent was smart enough to decide to do it. It's that nothing stopped it from being able to. The autonomy of the decision is almost beside the point once the agent is holding a key that opens the account, the disk, and the billing console. Scope the keys down first, and a huge share of the worst-case scenario disappears before you've written a single rule about behavior — because the capability to cause it is simply gone.
Permission scoping is the strongest control you have, but it doesn't cover everything, because not every rule is really an access rule. Some of what you need to constrain isn't "can this agent touch this resource" — it's "is this specific action, right now, within the shape we agreed to." An agent might legitimately need the ability to call an external API — but you still need "no more than once an hour, no matter what the agent decides the situation calls for in the moment." It might legitimately need write access to a codebase — but only within one narrow directory, never outside it, on every single run. Access control answers whether. It doesn't answer how much, how often, or under what condition. For that, you're back to the fence: deterministic code that sits outside the agent's own judgment and checks what actually happened — not what the agent says it did, not what it planned to do, not its own account of its own behavior. After the fact, against the real output, with no vote from the thing being checked.
Here's the good news: for a whole category of these fences, someone already built them for you, the same way scoped IAM roles and least-privilege credentials are already a solved problem in identity platforms. "The output has to be valid against this schema" — a validator exists. "This code has to type-check" — a compiler exists. "This function can't exceed this complexity" — a linter exists. If your rule is generic enough that it applies to any codebase, any identity, or any output shape, it's probably already solved, and wiring it in is an afternoon.
That's the easy 20%. Here's the part nobody sells you a library for.
Almost none of the rules that actually matter to your business are generic. "This workflow may call that API at most once an hour, no matter what the agent decides in the moment." "This agent may only ever act inside this one narrow slice of the system, full stop, even on the actions its own credentials technically allow." These aren't type errors, they're not access-control questions, and they're not style violations. They're specific to your business, your risk tolerance, your promises to your customers — and there's no off-the-shelf tool that knows what those promises are, because you're the only one who made them.
Which means you have to write the fence yourself. In real code. Running outside the agent, after the agent, checking the actual result against the actual rule, with the power to stop the outcome cold if it doesn't match — not politely note the mismatch and let it through anyway.
That is the real work of building an agentic system a business can actually rely on. Not a better prompt. Not a longer instructions file. Not even, on its own, a tighter permission boundary — though that's the first and cheapest thing to get right. It's turning every promise you've made, one rule at a time, into either a door that doesn't exist or a check that can't be talked out of doing its job.
And that's the actual argument against full autonomy today — not that the models aren't capable enough, but that most of us haven't done this harder, less glamorous work yet. We've scoped some permissions. We've written some sentences. We haven't finished building the fences for the behaviors that access control alone can't cover. An agent given wide permission and zero custom enforcement isn't autonomous — it's unsupervised, and there's no way to tell the difference between "trustworthy" and "hasn't failed yet" until something is actually checking.
So keep the agents finite. Keep the workflows finite. Not forever, and not because the ceiling is fixed — but because finite is the size of surface you can actually afford to lock down and enforce for right now. Grow the freedom exactly as fast as you grow the locks and the fences, and not one step faster.
None of this holds if it only lives in someone's head, or in a single conversation with an agent that nobody else was in the room for. A decision about what your agents are and aren't allowed to do has to survive the person who made it going on vacation, changing teams, or just forgetting the exact reasoning six months from now. So write it down as a decision of record — not a comment buried in a file, but a document your whole team can point to: here's the rule, here's why we made it, here's what it costs us if we're wrong. That's still a sentence. It's just a sentence with an address, one that outlives the conversation that produced it.
Then give that sentence a body. For every rule that matters enough to write down, write the task that checks it — the actual command that runs, the actual check that fails the build, the actual gate that blocks the push if the rule was broken. Not "we agreed to this," but "here's the command that proves it's still true, right now, on this change." A rule with no task attached is a decision nobody's actually keeping. A rule with a task wired to it enforces itself, every time, whether anyone remembers to check by hand or not.
That's the discipline underneath everything above: build your projects to hold the decisions, the rules, and the policies you make about your agents — and build the tasks that validate every one of them before the code ever ships. Not after a postmortem. Not when someone happens to notice. Before it leaves your hands.
An agent that doesn't have permission, can't. A rule that doesn't have a task behind it, doesn't count. Everything else is just a sentence, waiting to be broken by something that was never told to care.
Stay Updated
Get notified when I publish new articles on development, architecture, and tech.