Point of view
Four ideas I keep returning to
These are positions, not predictions. Each one carries the strongest objection I know of, stated in the same place — because a position that hides its counter-argument is not a position, it is an advertisement.
01
The governance gap is real, and under-discussed
Your company knows exactly who is allowed to approve a £2m payment. Delegated authority. Segregation of duties. A signature that means somebody's name is on it. It has no equivalent answer for the agent that just raised one.
What is called AI governance today is mostly monitoring and reporting — a careful account of what has already happened. Useful, but an audit log is not a control; it is a record of the moment the control was missing. The interesting question is what a system is structurally permitted to do before an action commits.
Where this is weakest
Knowing the gap exists does not tell you how wide to open the gate. That is a harder problem, and I do not think anyone has solved it well yet.
02
You cannot fix a probabilistic system by asking it to be more careful
When a model does something it shouldn't, the fix is almost always a line added to the prompt. It works. Mostly. Which is the problem — a rule that holds 98% of the time is not a rule, it is a tendency, and you do not get to choose when the 2% arrives.
Making the model more capable does not close this. Each improvement makes the failure rarer, later, and more expensive, because by then you have built on top of it. Reliability has to come from outside the probabilistic layer.
Where this is weakest
This is a design argument, not a scepticism-about-AI argument. I am emphatically pro-AI, and I think most organisations are being too timid rather than too bold. The way you earn the right to be bold is by putting the reliability somewhere the model cannot reach.
03
The bottleneck is almost never the technology
It is the data nobody profiled. The tacit organisational rule nobody wrote down. The incentive nobody checked. And the question of who owns the outcome when it goes wrong.
Every one of those is an organisational fact, and every one is discoverable before a line of code is written — by asking. Technology rarely fails for technical reasons.
Where this is weakest
This is the idea I have built a framework around. It is also the idea I am most often told is obvious — and most rarely see acted on.
04
Second-order consequences deserve attention
If AI handles the routine work, human judgement stops accumulating experience. Expertise is built from the boring cases, not the interesting ones — and what fills the gap, when the boring cases stop arriving, is mediated narrative rather than contact with reality.
I raise this about the thing I am helping build, not about someone else's.
Where this is weakest
I have no method for this one. I am not sure it is even a solvable problem rather than a cost to be understood and accepted. But it should be said out loud by the people building the systems, and mostly it isn't.