|
[Webinar] 8 levels of context maturity in AI-native engineering (Sponsored)
 |
AI is in your engineering workflows. While the token spend shows it, the throughput doesn’t. The human is very much still in the loop, and that’s a context problem.
Join live July 23 (FREE) to learn:
The 8 levels of context maturity: where most teams are stuck and what the ceiling looks like at each stage
Why more MCPs, rules, and skills provide agents access but not understanding
How leading engineering teams are using a context engine to make the most of their agents
There is a persistent gap between the AI agents that perform well in a demonstration and the ones that hold up under real production traffic.
This is because the systems handling live customer workloads tend to depend on the language model far less than their presentation suggests. Most of their behavior runs through conventional, deterministic code, with the model invoked at a small number of specific decision points. The difference between a dependable agent and a fragile one often comes down to a set of engineering decisions about where the model is given control and where it is held back.
But how do you build agents that work reliably in production? Are there any best practices?
In 2011, Heroku’s co-founder published the Twelve-Factor App, a set of principles for building web services that deploy and scale reliably, and that document influenced how a generation of engineers thought about software.
A parallel effort later appeared for AI in the form of a widely shared set of principles called the Twelve-Factor Agents. These principles were aimed at making language-model applications robust enough for production. The specifics have kept evolving since, and teams at companies like Anthropic, Cognition, and Intercom have added lessons of their own.
 |
In this article, we try to explore the collective thinking into a smaller set of practices and explain the reasoning behind each one, rather than asking anyone to memorize a numbered list. The practices fall into four areas, with a final section on the tradeoffs. Here’s what we will cover:
How does context control what the model sees on every call?
How does the control flow keep the loop and its stopping conditions in deterministic code?
How does the state hold memory in software while the model stays stateless?
How does the scope keep each agent narrow and supervised?
By the end, the article will have built a working definition of a production agent and a clear view of which decisions carry the most weight.
Disclaimer: This post is based on publicly shared details from various sources. References at the end. Please comment if you notice any inaccuracies.
The Loop
Let us start with the simplest agent anyone can build, because it reveals the structure that everything else refines.
At its core, an agent is a loop. The model receives some context, which is the running record of what has happened so far, and it returns a single decision about the next step to take. That decision arrives as structured output. It is machine-readable data, such as JSON rather than free-flowing prose, so that ordinary code can read it and act on it. The surrounding code executes the requested step, appends the result to the context, and returns everything to the model for the next decision. The loop continues until the model signals that the work is finished.
A useful way to picture the model’s role is as a function that starts fresh on every call.
On each call, the model sees only the context provided to it, makes one decision, and discards everything the moment it responds. The context window, which is the bundle of text the model reads on a given call, serves as the entire memory of the system.
 |
The appeal of this design is clear. A developer describes a goal and supplies a set of tools, and the model works out the order of operations on its own. The design requires less branching logic, since the model handles the decisions. This is the version of an agent that can perform quite well in a demo setup. However, the trouble begins when real traffic arrives.
Failure Modes
The design we looked at in the previous section fails under production conditions, and the failures follow some predictable patterns:
Compounding error multiplies across chained steps.
Confidently wrong output reaches a user.
A loop runs longer than intended.
State disappears when the plan lives only inside the model.