Securing AI-augmented development: the control that holds is imposed, not supervised

A previous piece in this series established a clear limit: having one AI review another AI’s code displaces the control without closing it. Two closely related models share blind spots, and the last reliable link remains a human eye.

The logical next step seems to be adding intelligence on top, a more capable supervising agent tasked with catching what the others let through. The intuition is common, and it is aimed at the wrong level. A supervisor that re-reads conclusions mostly validates the best-argued errors. What holds a chain together, where AI produces a growing share of the code, plays out elsewhere: in a control architecture that the system imposes and that no agent can bypass.

The distinction is concrete. A control that an agent chooses to run depends on that agent’s state, its context, its instructions, its load. A control that the system runs depends on none of these. The whole reliability of AI-augmented development plays out in that gap.

Reading a line of reasoning is not measuring it

A line of reasoning is read; an effect is measured. These are two different operations, and confusing them is what founds the illusion of the supervisor.

Take a maintenance operation carried out across a fleet of sites. The script reports success: every server responds, the expected status code comes back, the output check is green. Except that the observed signal, the server’s response, is blind to the real damage. On several sites, the application has lost the right to write to its own file system. The control was measuring one thing, the server responds, while believing it measured another, the operation succeeded.

A few hours later, the incident surfaces through a user. An analysis session, without the context of the original operation, reconstructs a plausible and coherent cause. It is wrong. What overturns it is neither a finer piece of reasoning nor a review by a more capable instance, but a dated fact, read off the system, the modification time of a configuration file, which contradicts the hypothesis.

A supervisor that had merely re-read this analysis would have validated it. It was well built. What killed it was a measurement, not an authority. Stacking intelligence on top of a chain never turns a review of reasoning into a verification of fact. A control that reads is worth less than a control that measures.

The control that holds is not the one an agent chooses to apply

Grant this limit and ask the real question: which control holds when the agent meant to apply it is itself fallible?

A review agent, however well briefed, remains something a session invokes. It shares the condition of any session: it can be launched without the right context, on incomplete instructions, or simply skipped because the operation seemed harmless. Its protection depends on the discipline of whoever calls it. This is a control by convention, and a convention gets skipped. It gets skipped all the more easily because nothing, in execution, makes it mandatory: it lives in the instructions given to an agent, not in the mechanism that runs its actions.

The control that holds is of another kind. It is the one the system runs, not the one an agent decides to run. In an agentic chain, this layer exists: the environment that frames the agents can refuse a command before it leaves, independently of what the agent understood of the situation. A rule of this kind is not forgotten, not bypassed, and does not depend on the state of a session. It applies, full stop.

The gap between the two is quickly measured on a real configuration. Take an agent chain fitted with several hundred permission rules and no denial rule. The reading is deceptive: it looks like a rich setup. In practice, an irreversible command on the production directory, a recursive change of ownership, a mass deletion, passes without the slightest friction, in any session, at any hour. The absence of denial rules is not a configuration detail, it is the hole through which the incident arrives.

What a technical leadership recognises immediately is the underlying principle: least privilege, and non-bypassable controls. Neither depends on the intelligence of the agents. Both depend on what the system permits them, and what it refuses them. The question of who, in the organisation, decides these rules and answers for them belongs to another plane, that of the governance of autonomous agents. The present argument stays on the mechanism: what the machine allows, and what it blocks, before any human decision.

The perimeter has not disappeared, it has changed layer

This reasoning has a precedent, and naming it helps place it. For twenty years, application security played out at the perimeter: a firewall, network rules, a boundary that nothing running inside could argue with. The control was deterministic and imposed by construction. Nobody asked a machine whether it agreed to respect the firewall.

The anchor of this series set out the shift under way: the risk has left the network to enter the code, because the software factory now produces, through AI, a share of that code without a human having written it line by line. The conclusion drawn too quickly is that discernment is therefore needed inside, an intelligence able to judge what is happening. That is to repeat the supervisor’s error, one layer down.

The right conclusion is that the logic of control follows the same shift as the risk. It is not abandoned in favour of “intelligent” supervision, it changes layer. What was imposed on the network must be imposed at the agents’ execution layer: what an agent may read, write, trigger, and above all what it may not. The firewall did not judge intentions, it applied a rule. The agentic layer calls for the same kind of rule, as close as possible to the action.

One last trait of the network model is worth keeping: the control bore on a fact, not on a declaration. An agentic operation needs the same anchoring. Checking that a server responds does not say the operation succeeded; checking that the application can effectively write where it must write, does. The success criterion must bear on the real effect, not on a signal that resembles it. This is the point of contact with reality without which a control, however automated, produces only plausible validations.

This point is worth holding firmly, because automation makes it fragile. The more a chain links steps together without ever touching a fact, the faster it produces coherent and unverified conclusions. A control wired to reality does not go faster, it goes more accurately: it stops the chain when the expected effect is not there, instead of letting it run on a signal that reassures. The speed of an agentic chain is only worth something if something, somewhere, checks that it is moving in the right direction.

Design the control before strengthening the intelligence

From all this an order of priority emerges, and this is where the decision lies.

An agentic chain is not made reliable by placing a more intelligent AI at its summit. It is made reliable by first designing the architecture that constrains it. This architecture rests on three pieces, none of which depends on the capacity of the models. Deterministic guardrails, run by the environment that frames the agents and not by the agents themselves, which make irreversible operations impossible without explicit validation. Success criteria wired to the real effect rather than to an indirect signal. A memory of operations shared across sessions, so that the lesson drawn from one operation is legible to the next, even when it arrives after the fact.

These three pieces share one trait: each is a place where the chain meets a fact rather than a declaration. The guardrail imposes a rule, the criterion measures an effect, the memory carries forward a verified lesson. It is this accumulation of points of contact with reality, and not the finesse of a supervisor, that holds the whole together.

In this architecture, a supervising AI has a place, but a precise one. It is only useful if its mission is to go and verify reality, not to re-read a line of reasoning. An agent tasked with cross-checking an operations log against the observed state of a fleet produces value, because it confronts two facts. An agent tasked with validating another agent’s conclusions does no more than any text reviewer would: it validates text. The difference is not in the supervisor’s intelligence, it is in what it is given to examine.

That leaves a single question to ask in front of any AI-augmented development chain, whether internal or supplied by a third party. Not: is the model intelligent enough not to make mistakes. But: what does the system impose that no session can bypass, and at what moment is each operation confronted with a measured fact rather than its own declaration of success. The answer to that question tells the real solidity of the chain. The intelligence of the models does not.