Progressive disclosure: giving AI the right context without making it read everything

The anchor article of this series made one observation: an agent only has what is in its context, and much of what makes a business application correct is written nowhere. The most common reaction is to write a great deal, then load all of it. An instructions file that grows with every incident, a project document pasted in full at the start of each session. The result almost always disappoints: the agent that knows everything behaves like the one that knows nothing. It ignores the rule that mattered, contradicts a convention set three pages earlier, rewrites what already existed.

The problem is not the amount of documentation but how it is delivered. A knowledge base that is useful to an agent is not loaded, it is routed. Each module carries a short description, always visible, that says what it contains and when to read it; the detail only enters the context for the module the task calls for. The principle has a name, progressive disclosure, and it comes from interface design, long before it served language models.

This article describes the two ways an agent loses its footing when context is badly served, shows where the principle comes from and how recent skills mechanisms make it concrete and measurable, then applies it to the documentation of a business project in five moves. It closes on what routing does not guarantee, and on what a CIO (chief information officer) can demand of their documentation before connecting an agent to it.

Two ways to lose an agent: context starvation and context indigestion

The first way is familiar. Little or nothing is written down, and the agent infers from the code what the code does not say. It then produces something plausible and wrong: it allows a validated quote to be edited because no rule in its context forbade it, it bypasses a regulatory constraint it has never seen, it redefines the exchange contract with a neighbouring system because that contract lived in a developer’s head and had never been made explicit. The resulting code is consistent with itself and inconsistent with the system. This is the failure mode the anchor article of this series described in detail.

The second way is the opposite reflex, and it is less recognised because it looks like diligence. The team documents everything, then loads everything. The instructions file at the root of the repository gains a line per incident and reaches 40 pages. The project documentation is pasted in full into every session. Three effects follow. Cost, first: every task pays to read the whole, including the nine tenths that have nothing to do with it. Attention, next: the work of Liu et al. showed that a model’s performance drops sharply when the useful information sits in the middle of a long context, and Anthropic’s engineering post on context engineering describes an “attention budget” that runs down as the context grows. Contradiction, finally: a 40-page document contains at least one obsolete rule, and the agent reads it with the same confidence as the current one. It has no way of knowing which of two contradictory conventions is today’s.

The two failure modes share a root. In both cases, the knowledge is not where the agent needs it, at the moment it needs it. Context is a budget, not a bag into which everything that might be useful is poured. Every token spent reading a convention unrelated to the task is a token not spent reading the code being modified. The useful question is therefore not “have we documented enough?” but “does the right page arrive at the right moment, and only that page?”.

An interface principle that is 20 years old, and the mechanism that makes it measurable

The answer to that question has existed for a long time in another discipline. In 2006, Jakob Nielsen formalised progressive disclosure as the answer to a classic tension in usability: users want both powerful features and a simple interface. The principle has two steps. Show only a few options at first, the most important ones; offer the rest on request. The interface stays readable, the power stays reachable, and the user decides when to go one level down.

The transposition takes little effort. The reader is now an agent. The screen is the context window. The advanced options are the documentation modules. What stays permanently visible is small and decisive: enough to choose what to open, not enough to do the work. What is heavy only appears after a decision to open it. A principle designed to spare human attention applies almost word for word to a system whose attention is, likewise, a counted resource.

What has changed recently is that a tooling mechanism makes the principle concrete and quantified. The skills documented by Anthropic work on three levels. At the first level, only each module’s metadata (a name and a description) is loaded at start-up, at around a hundred tokens per module. At the second level, the body of the module only enters the context when the request matches its description, at under 5,000 tokens. At the third level, the accompanying resources (scripts, templates, references) cost nothing until they are explicitly called. The documentation states that the description must say both what the module does and when to use it, because that is what the model compares the request against when deciding whether to open it.

Read with an engineer’s eye, this mechanism is a three-tier index. The point that matters is this: the entry metadata does the routing, never the body of the document. A badly written description sends the agent to the wrong place; a body loaded without need squanders the budget. The same logic appears, under other names, in the trigger-based rules files of most coding assistants, in the shared instruction-file conventions that span tools, and in what Anthropic calls “just-in-time” retrieval: the agent keeps lightweight identifiers (a file path, a reference) and only loads the content when it is about to use it. The principle belongs to no vendor. What varies is whether the tooling enforces it or whether the team has to build it into its own documentation.

Applying it to project documentation: the modular, self-routing knowledge base

The documentation of a business project is not a set of skills, but it can borrow the same architecture. Five moves are enough to turn a monolithic corpus into a knowledge base the agent routes itself.

The first move is to split by decision domain rather than by file type. An agent in the middle of a task asks itself questions of a specific kind: which business rule applies here, which architectural boundary must not be crossed, which coding convention is in force, which contract binds me to the neighbouring system, which data is sensitive, how does this component deploy. Each kind of question becomes a module. Business rules and invariants, architecture and boundaries, conventions, integrations and external contracts, security and data, operations. The split follows the decisions the agent will have to make, not the folder tree of the repository.

The second move standardises the header of every module, because the header is what routes. A name. A one-sentence description built on the pattern “read when”, for example “invoicing rules: read before any change touching quotes, credit notes or discounts”. A scope, a status with a last-review date, an owner. Everything else in the module can be long, technical, dense. The header must be readable in three seconds and trigger the right decision to open. A complete header fits in five lines:

Module: invoicing rules
Read when: any change touching quotes, credit notes or discounts
Scope: amount calculation, accounting exports; excludes payment handling
Status: current, reviewed on YYYY-MM-DD
Owner: back-office team

The third move keeps the root index short. The instructions file at the root of the repository, whatever its name in a given tool, now contains only two things: the cross-cutting invariants that apply to every task, and the list of module descriptions. A useful order of magnitude is one screen, readable by a human in a minute. Everything else is one link away. If the index overflows, a module is missing or a description is too long.

The fourth move references the heavy appendices without ever inlining them. Data schemas, API (application programming interface) contracts, architecture decision records (ADRs), incident logs: all of this exists, is pointed to from the relevant module, and only enters the context if the module calls for it. This is the third tier of the index, the one that costs nothing until it is opened.

The fifth move tests the routing. Give the agent a representative task and watch what it opens. If it opens the wrong module, or none, the description is at fault, not the agent. Repeat the exercise on a dozen tasks typical of the project. It is the exact equivalent of a usability test, with a user who leaves a complete trace of their path.

A recent example brings the whole together. A professional software product delivered as a service (SaaS), in its launch phase, had accumulated over its build several dozen pages of project documentation, complete and little used. The team rebuilt it along these five moves: a root index, modules by domain, a one-line description per module. The effect was visible without instruments. The agent stopped rereading the whole at every session and started opening the module the task called for; conventions stopped being rediscovered, then contradicted; and the documentation came back to life, because every module now had a precise reason to be opened, and therefore corrected.

What the team gains, beyond the agent

The agent is not the first beneficiary. A routable knowledge base is first of all documentation that humans read. The developer joining the project scans the index, opens the module that matches their ticket, and finds the rule and the reason for the rule at the same time. They come in through the same door as the agent, and that door has been tested.

The description becomes a contract. Writing “read when” forces the author to say what the module is for. A module for which nobody can formulate the “when” has no reason to exist on its own: it merges with another or disappears. This constraint prunes documentation far more reliably than an annual review.

Freshness follows the same movement. A module that gets opened gets corrected, because the error is seen at the moment it costs something. The status and review date carried in the header make staleness visible before it is read. Monolithic documentation, by contrast, is corrected by nobody because it is read by nobody: its volume protects it from reading, and therefore from correction.

Many teams already practise this pattern for humans without having named it. A framing note that points to the right corpora instead of reproducing them, summary sheets that condense and direct, a register of architecture decisions consulted before every change: that is progressive disclosure, applied to human readers. Agents invented nothing. They made the absence of this pattern expensive and its presence measurable, in tokens consumed and tasks completed. Naming it makes it reproducible from one project to the next, and auditable.

What routing does not guarantee

A badly written description sends the agent to the wrong place, and nothing signals it. The agent does not know what it did not open. This failure mode is more insidious than an outright error, because the work produced looks complete. The quality of descriptions is therefore a review object in its own right, on a par with code.

A stale module misleads with confidence. Progressive disclosure reduces the volume read; it does not validate what is read. It calls for minimal governance: an owner per module, a review cadence, a description versioned with the code it describes, and a review of module changes as demanding as the review of code itself.

The agent may decide not to open a module. Routing rests on the model’s judgement in front of a description; it is a control the agent chooses to apply, not a control the system imposes. An earlier article on this blog set out the difference between these two kinds of control, and it applies here without reservation. For everything that must hold whatever happens, a security rule, an irreversible operation, a regulatory constraint, the knowledge base is not the mechanism. Tests, the checkpoints of the continuous integration pipeline and permissions are.

Finally, organising knowledge does not create knowledge. A module can only encode what someone has made explicit, and making things explicit remains the most underrated skill in the trade. Progressive disclosure is a delivery discipline for knowledge that exists. It replaces neither the workshop where business rules are formulated nor the review of the code the agent produces. Its place is elsewhere: it makes the other controls cheaper, because a well-routed agent makes fewer avoidable mistakes, and tests and review are left to catch only what remains.

What a CIO can demand of their documentation before connecting an agent to it

Four requirements sum it up, and each can be checked in a single session.

A root index that fits on one screen, where each line says when to open the module it describes. Modules split by decision domain, each with an owner and a review date readable in the header. A clean separation between what is always loaded, what loads on demand and what is merely referenced. And proof of routing: on the project’s everyday tasks, the agent opens the right module, and that proof is replayed at every overhaul of the documentation.

This article extends the analysis of the series anchor: an AI prototype holds up in a demo and stalls in production because the engineering layer that keeps a system standing has yet to be built. Documentation that knows how to say when it should be read is one piece of that layer, the cheapest to put in place and often the first one missing.

Documentation an agent can route is documentation a team can take over. It is the same test, and it happens before the first line of generated code.