Article · Governance architecture series
The physics of an autonomous decision
Why governing what an AI system says is not the same as governing what it does — and what it actually takes to close that gap.
Enterprise AI crossed a threshold quietly. Systems that once drafted text now approve refunds, issue quotes, escalate claims, and modify customer records. The capability to act arrived years before the authority structures needed to govern that action.
This isn’t a failure of any individual tool. Language models became capable of using tools. Tool use became agency. Agency reached the systems where consequences actually live — the CRM, the policy administration system, the payment rails. At every step, the question “is this action permitted, from this state, right now?” was answered locally, partially, or not at all.
Most organisations have elaborate machinery for controlling what an AI system says. Almost none have anything for controlling what it commits.
§ 1
What happens when a regulator asks
Picture the question a regulator, an auditor, or a board eventually asks: “For every automated decision made on disputed accounts last quarter — what was the state of each account at the moment of decision, which rules applied, and who approved anything above the threshold?”
In most organisations, this question cannot be answered cleanly. The evidence is fragmented across applications, each with its own partial log, none capturing what the system actually knew at the moment it acted. Teams spend days reconstructing a picture that remains incomplete by design — because no single layer was ever responsible for the decision as a decision.
§ 2
Govern the entity, not the tool
Most governance attempts anchor to the wrong thing. They govern the model, the application, or the conversation. But the consequence never actually lives in any of those. It lives in the entity — the customer account, the deal, the claim, the contract.
An entity is touched by many systems over its life. If each system enforces its own version of the rules, the entity ends up governed by whichever rules the weakest caller happens to apply. Anchor governance to the entity itself instead, and every caller — present and future — is governed identically, automatically, by the same definition of what’s possible.
§ 3
What an authority layer actually requires
Six structural properties have to hold simultaneously. Partial combinations fail in specific, predictable ways — a guardrail without deterministic permission still lets the wrong action through; deterministic permission without evidence is unenforceable in front of a regulator; and evidence or permission without durability quietly expires while nobody’s watching.
01
Universal state awareness
One authoritative answer to what's true about this entity right now — regardless of which system asks, with no back door by which state changes unobserved.
e.g. a customer flagged for hardship shows that status to every caller, instantly — not just the system that flagged it
02
Deterministic permission
What's possible from a given state is defined by configuration and enforced structurally — never inferred, interpreted, or left to a model's understanding of the rules.
e.g. the same request, on the same account state, produces the same answer — every time, on any machine
03
Mandatory evaluation
Every proposed action passes a defined sequence of checks before it commits. No caller — human or machine — decides which checks apply, or when one may be skipped.
e.g. there is no code path, and no override, that lets an action reach production unevaluated
04
Designed human checkpoints
Where the organisation has designated judgment to a person, the flow routes to that person as a built-in checkpoint — not as a failure escalation bolted on after something went wrong.
e.g. above a threshold, the action has no path to completion without a named approver
05
Permanent, independently verifiable evidence
Every decision — permitted, held, or refused — produces a record of what the system knew, what rules were in force, and who decided, that a third party can verify without taking the enterprise's word for it. A tamper-resistant log inside a system the same party controls is still a self-attested claim, not a verified one.
e.g. a regulator can confirm the record hasn't been altered independently of trusting the enterprise's own operational integrity
06
Durable authority, continuously reassessed
A permission granted once is not permission forever. Where an agent holds standing authority — a threshold, a recurring action, anything whose effects continue after the moment it was granted — the justification for that authority has to be reassessed on some cadence, not left standing until someone remembers to check it.
e.g. an agent authorised to auto-approve claims under a threshold has that authority re-validated against current risk data, not just checked once at setup
One honest caveat, stated plainly: these are the properties of a technical substrate. They are necessary engineering conditions. They are not, by themselves, sufficient to satisfy a regulator or auditor. Frameworks like the EU AI Act and ISO/IEC 42001 also require a documented risk assessment, a named accountable owner, evidence of a conformity process, and proof that the humans in the loop were competent for the role — not just present. An organisation can build all six properties correctly and still fail an audit, because the audit checks the organisational apparatus around the control, not only whether the control itself works.
A structural gate is a necessary engineering condition. It is not, by itself, a regulatory one.
§ 4
A test worth running on your own AI estate
You don’t need architecture diagrams to find out where you stand. Six honest questions do most of the work.
Pick your three highest-consequence AI-driven actions. For each, can you name who is structurally required to approve it before it happens — not who's supposed to review it after?
Pick one entity your AI touches often — a customer, a claim, a contract. How many different systems can act on it? Do they all see the same current state?
Ask what happens at 11pm on a Sunday. If every human went home right now, which actions would still be permitted to execute — and did anyone decide that on purpose?
Find last quarter's disputed-account decisions. Can you produce, today, what the system knew and which rules applied at the moment each one was made?
Check who owns the rules. Can the people accountable for a rule change it directly — or does every update require an engineering ticket?
Pick one standing permission your AI holds. When was it last re-validated against current data — or has it been running unchecked since the day it was granted?
If more than one of these produces a shrug rather than an answer, the gap isn’t in your AI. It’s in the layer that was supposed to sit underneath it.
Addendum — August 2026
What changed after this was published
This article generated real pushback worth reflecting here rather than editing away quietly. Several credentialed readers converged independently on gaps the original version didn’t cover.
The evidence property originally said “immutable.” The correction, and it was a fair one: immutable inside a system the enterprise itself controls is still a self-attested claim, not a verified one. The property now specifies independently verifiable evidence — the same discipline already applied everywhere else, pointed at the evidence layer itself.
Two readers, independently, identified that nothing in the original five properties addressed what happens to a standing authorisation after it’s granted, if the facts underneath it change while it’s still in effect. That’s now its own property — durable authority, continuously reassessed.
A third reader made a fair and important distinction: everything above describes a technical substrate. It does not, on its own, satisfy a regulator. That caveat is now stated plainly in the text, rather than left implicit.
None of this weakens the argument. It’s the argument holding up to scrutiny, and getting more precise because of it.