I spent years on security and identity products before I spent years on AI ones, and the transition has been strange in a specific way. Everything the security world learned the hard way about delegated authority — who is acting, under whose permissions, with what record — is being rediscovered from scratch by teams building agents, usually about six weeks before a deal review where a customer's security architect asks the obvious question.
The pattern is consistent enough to predict. A team builds an agent that does something genuinely useful. It demos beautifully. Then it meets an enterprise buyer, and the conversation is not about capability at all. It's about what happens when the agent is wrong, who could have seen it coming, and how you put things back. If the answers are being invented in that meeting, the deal slows down by a quarter.
Why agents break the old permission model
Traditional software has a comfortable shape: a user makes a request, the system acts as that user, within that user's permissions, in a session that user is watching. Authorization is a check at the boundary, and the human is right there.
Agents dissolve most of that. They act across multiple systems in a single logical task. They run asynchronously, sometimes on a schedule, sometimes on behalf of a user who left hours ago. They compose tools in orders nobody enumerated in advance. They may act for a user, for a tenant, or for the platform itself, and those are three very different things with three different blast radii.
A permission model designed for request-and-response doesn't cover any of that. Not because it's badly built, but because it was answering a question that no longer describes the system.
The question that unblocks an enterprise AI deal is rarely “is the model good.” It's “what happens when it's wrong, and who can see it.”
Three surfaces worth designing on purpose
Identity and delegation. An agent should have an identity of its own, and it should never be able to exceed the authority of whoever it's acting for. That sounds obvious and it is routinely violated, usually by an agent running with a broad service account because that was the fastest way to make the demo work. The moment an agent can do something its principal couldn't, you have built a privilege escalation path with a friendly interface. In multi-tenant systems this is not a nuance, it's the whole ballgame.
Auditability. An audit record for an agent is not a log line saying an action occurred. It's the reasoning trail: what the agent was asked, what it retrieved, which tools it called with which arguments, what came back, and what it decided as a result. This is the same artifact that makes the system debuggable for your own engineers, which is why I'd argue for building it even if no customer ever asked. The customer asking is just the forcing function.
Reversibility. Undo has to be a designed capability, not a support process. For every action an agent can take, someone should have answered: can this be reverted, by whom, within what window, and does reverting it leave a record too? Actions that genuinely cannot be undone — sending money, deleting data, notifying a customer — belong in a different tier with a different approval posture, and that tiering is a product decision made at design time, not a policy written afterward.
The commercial argument
Here's the part I think gets missed. Governance isn't a tax on autonomy — it's the currency you buy autonomy with.
Every increment of independent action you want your agent to take gets paid for with an increment of visibility and reversibility. A customer will let an agent do very little if they can't see what it did. They'll let it do a surprising amount if they can watch it, constrain it, and put things back. The teams shipping the most autonomous agents in enterprise environments are not the ones with the best models. They're the ones whose customers trust the controls.
Which means governance work expands your roadmap rather than constraining it. Retrofit it and you spend the following year removing capability to fit a model that wasn't designed for it — narrowing scopes, disabling actions, adding approval steps that make the product feel worse than the manual process it replaced. Design for it and each new tier of autonomy is an increment rather than a renegotiation.
What this looks like in practice
Mostly it looks like asking three questions during design rather than during review. On whose authority does this run, and can it exceed that authority anywhere? What record does this leave, and would that record satisfy someone investigating a bad outcome six months from now? What's the undo, and who's allowed to press it?
None of those are hard questions. They're just embarrassing to answer late, and expensive to retrofit — which is a reasonable working definition of a feature that should have been in the spec.