At turn 4 the agent is sharp. At turn 40 it is still producing answers, the answers are still plausible, and the system is silently wrong. No exception. No 4xx. No trace entry. The agent degraded and nobody noticed until the damage compounded.
This is not a model problem. It is a context problem. And most engineering teams have no framework for it.
The production gap prompt engineering cannot close
LangChain's 2026 State of Agent Engineering puts 57.3% of respondents with agents in production (67% at large enterprises). The same report names "context engineering difficulties at scale" and "managing context window limitations" as explicit top failure modes. Quality (accuracy, consistency, and brand adherence) is the single largest barrier at 32%. Model capability does not make the list.
That finding deserves a moment. The teams shipping agents are not blocked on what models can do. They are blocked on what they put into the context window and how they manage it across turns.
Anthropic's applied AI team draws the boundary precisely. Their published guidance defines context engineering as: "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference," positioning it as the evolution beyond prompt engineering. The distinction matters. Prompt engineering is one input to one inference call. Context engineering is the entire informational state across every turn: system prompt, tools, message history, retrieved memory, injected state, and the compaction decisions that govern all of the above.
A prompt engineer writes good instructions. A context engineer decides what the model knows at every step, in what order, at what level of detail, and what gets discarded when the window fills.
The transformer attention mechanism turns context management into a compounding problem. Every new token carries a computational cost proportional to every existing token. Vishnyakova (arXiv:2603.09619) makes the level-1 framing explicit: in multi-turn environments, prompts are necessary but not sufficient. The architecture of the context window is the load-bearing concern.
Three silent failure modes with no exception class
None of the following failure modes throw an error. They produce plausible, slightly degraded output that compounds over subsequent turns.
Constraint erosion is what happens when the operating instructions that govern an agent's behaviour live in the early turns of a long context. As the window fills, attention weight on those early tokens decreases. Bousetouane (arXiv:2601.11653) formalises this: agents lose constraint adherence as interactions extend, errors accumulate, and the system drifts from its intended operational envelope. The agent does not know it has drifted. The monitoring dashboard shows normal response latency.
Context poisoning happens when stale, incorrect, or adversarially-shaped entries persist in the context and influence subsequent reasoning. Bousetouane's Agent Cognitive Compressor (ACC) study identifies transcript-replay memory as the primary vector: when an agent's memory is implemented by replaying the raw conversation transcript, every error the agent ever made re-enters the context on every subsequent turn. An early hallucination is not a one-time cost. It is a recurring tax.
Hallucination compounding closes the loop. Each hallucination that makes it into context becomes a premise for the next inference call. The model is not fabricating facts from nothing; it is reasoning coherently from a corrupted record. Vishnyakova (arXiv:2603.09619) formalises the failure taxonomy and presents the Klarna case study as an example of dual deficits: failures in both contextual design and intentional design that compound across a customer-facing agent's operation.
Anthropic's team names the overall pattern "context rot": performance degrades as tokens grow, and the degradation is a gradient, not a cliff. That is precisely what makes it invisible. A cliff throws an exception. A gradient produces slightly worse outputs, session after session, until the accumulated drift becomes a visible incident.
There is a second failure mode at the retrieval boundary. Databricks identifies the cold-retrieval problem: "The agent must decide what to ask its memory store before it knows what is in there." Bigger context windows are not the fix. As Databricks puts it, packing millions of raw tokens into a prompt increases latency, raises compute costs, and degrades reasoning quality. Selective retrieval into a managed context is the only path that scales.
If your agent has state-evolution faults of the kind described in our previous post on production failure modes, context poisoning is often the mechanism. The benchmark harness validates and retries. The context never accumulates errors. Production does neither.
Five context quality criteria your team is not testing
Vishnyakova (arXiv:2603.09619) proposes five testable properties of the context window at any turn. The quote that anchors the framework is direct: "Whoever controls context controls behavior; whoever controls intent controls strategy; whoever controls specifications controls scale."
Most teams are not testing any of these five properties systematically.
| Criterion | What it means | How to test it |
|---|---|---|
| Relevance | Every token bears on the current task | Ablation: remove a context section and measure whether task performance changes. If it does not, the section was noise. |
| Sufficiency | The agent acts correctly without hallucinating missing facts | Disable external tool calls; measure accuracy on tasks that should not require them. Hallucination rate rises when sufficiency is violated. |
| Isolation | No cross-task or cross-tenant contamination | Run parallel sessions from different users on the same agent. Verify that session A context never surfaces in session B responses. |
| Economy | Context is no larger than required | Profile token counts per turn. Bloat is a compounding attention tax that degrades every subsequent call. |
| Provenance | Every entry is traceable to its source, timestamp, and injection mechanism | Attempt to reconstruct where any given context entry came from. If you cannot, you cannot find poisoned entries when an incident occurs. |
Relevance and economy are the criteria teams most often think they address through prompt quality. They do not. A well-written system prompt can still be surrounded by irrelevant tool outputs, stale retrieved chunks, and compacted summaries that preserved filler instead of decisions.
Provenance is the criterion that makes everything else auditable. Without it, you have no mechanism to identify which turn introduced a poisoned premise, no way to replay a session with corrected context, and no artefact to show an auditor when the question is "why did the agent do that."
Three infrastructure patterns that enforce context quality
Good intentions do not produce good context. Enforcement requires architectural choices.
Proactive compaction on cadence, not on overflow. The worst time to compact a context is when the window is full, because by then the model is already reasoning from degraded state. Anthropic's team recommends compacting on a schedule: summarise conversation history periodically, preserving decisions and commitments, discarding redundant intermediate outputs. The principle behind this is the same as the one behind structured note-taking. The agent does not need a transcript of its reasoning. It needs a record of what it concluded and why.
Bounded state with separated memory concerns. Bousetouane's ACC framework separates artifact recall from permanent memory commitment. Transcript replay is not a memory architecture. It is the absence of one. The ACC replaces unbounded context growth with bounded internal state updates, producing "significantly lower hallucination rates and more stable behaviour in extended IT operations, cybersecurity, and healthcare workflows." The mechanism is cognitive compression: facts that have been decided are committed to structured memory. The transcript is discarded, not appended.
Episodic-to-semantic distillation with lineage. Databricks distinguishes episodic memory (raw interaction records) from semantic memory (distilled patterns and facts). Agents should query semantic memory by default. Raw episodic records are the audit trail, not the context source. The infrastructure requirements follow from this: unified storage that supports both structured queries and vector similarity search, full lineage tracking so any semantic entry can be traced back to the raw records it was distilled from, and row-level security so organisational scope does not bleed across tenant boundaries.
For multi-agent systems, Anthropic's guidance adds a fourth pattern: orchestrator context partitioning. Sub-agents pass condensed summaries upward, not raw conversation history. The orchestrator receives the conclusions, not the transcript. This enforces isolation and economy simultaneously. It also means a compromised sub-agent context cannot poison the orchestrator's view of the task.
Context engineering is an ownership problem
The technical patterns above are available. Most teams know some version of them. The reason context quality degrades in production is not ignorance. It is that nobody owns the cross-cutting contract.
Context sits at the intersection of prompt design, retrieval architecture, memory management, and orchestration logic. On most teams, those concerns are owned by different people with different backlogs. The prompt lives with the ML engineer. The retrieval system lives with the platform team. The memory layer is wherever someone bolted it on. Orchestration is the application layer's concern. The context window is where all four interests meet, and today at most teams, nobody owns that intersection.
Vishnyakova (arXiv:2603.09619) maps the maturity progression: prompt engineering (turn 0 instructions) gives way to context engineering (full-window management), which gives way to intent engineering (goal-level specification), which eventually gives way to specification engineering (formal contracts between agents). Most enterprise teams in 2026 are at level one or early level two. They have invested in prompts. They have not yet invested in context as infrastructure.
Vishnyakova cites Deloitte's 2026 projection that 75% of enterprises plan agentic AI deployment within two years while scaling has "surged and retreated" due to complexity.
The pattern matches what the LangChain survey shows: most teams can get an agent into production. Fewer can keep it reliable across extended sessions at scale.
The ownership gap is also a measurement gap. IBM Research (arXiv:2603.22083) demonstrates that context quality can be modelled and improved as a systems property. That is only possible if someone is measuring it. Today's monitoring stacks track latency, token counts, and error rates. They do not track context provenance, sufficiency violations, or constraint erosion across turns. The metrics do not exist yet because the ownership does not exist yet.
This is the version of the infrastructure problem that does not get named in sprint planning. It surfaces as unexplained quality degradation, escalating context window costs, and incidents where the postmortem ends with "the model behaved unexpectedly" because nobody can reconstruct what was in the context when it did.
Monday morning: start with observability, not architecture
Pick one agent running in production. Pull the raw context window at turn 20 for five recent sessions. Score each context entry against the five criteria: is it relevant to the current task? Does it provide facts the agent needs, or is it carrying forward noise? Can you trace where it came from and when? Is this session's context isolated from other sessions' data?
If you cannot score the entries because there is no provenance metadata, that is the finding. You do not know what is in your agent's context. You certainly cannot detect when it degrades.
The first deliverable is not a new memory architecture or a compaction algorithm. It is a context observability layer: structured logging that captures what entered the context at each turn, from what source, via what mechanism, at what token cost. Build that before the next feature. Without it, everything else is speculation.
The broader production operating discipline covers incident response, monitoring baselines, and rollback contracts. Context engineering is one column in that discipline, but it is the column that makes the others auditable. You cannot govern what you cannot observe. And right now, for most teams in production, the context window is not observable at all.
— Vishnyakova — Context Engineering: From Prompts to Corporate Multi-Agent Architecture (2026)Whoever controls context controls behavior; whoever controls intent controls strategy; whoever controls specifications controls scale.
The teams that close the 2026 production reliability gap will not do it by prompting harder. They will do it by treating the context window as infrastructure: owned, measured, and managed with the same discipline they apply to a database schema or a message queue.
References
- Anthropic Applied AI — Effective Context Engineering for AI Agents (2025)
- Bousetouane — AI Agents Need Memory Control Over More Context, arXiv:2601.11653 (2026)
- Vishnyakova — Context Engineering: From Prompts to Corporate Multi-Agent Architecture, arXiv:2603.09619 (2026)
- Yang et al. — A Context Engineering Framework for Improving Enterprise AI Agents based on Digital-Twin MDP, arXiv:2603.22083 (2026)
- LangChain — 2026 State of Agent Engineering (2025)
- Databricks AI Research — Memory Scaling for AI Agents (2026)