Context compression for AI agents

An AI model can only read a limited amount of text at once, and every word it reads costs money. Context compression keeps that text short: old conversation turns are summarised, raw results that are no longer needed are removed, and the key facts stay. A customer-service assistant working through a long case can then keep going without losing the thread or running up the bill.

Why do AI agents need it?

An AI agent adds every tool call and its full result to its history, so the history grows with each step. Context windows "generally top out at around 1,000,000" tokens, Simon Willison wrote in March 2026, "and benchmarks frequently report better quality results below 200,000". Quality drops well before the window is full. Liu et al. (2023) found that models use information in the middle of a long input worse than at its start or end, and Anthropic describes context rot: as the number of tokens grows, the model recalls less of what is in its context. Providers also charge per token.

How does it work?

Anthropic's Effective context engineering for AI agents (September 2025) describes the main techniques:

• Compaction: when a conversation nears the limit, the model summarises it and a new context starts from the summary. Claude Code keeps architectural decisions, unresolved bugs and implementation details, drops redundant tool output, and continues with the summary plus the last five files it accessed.
• Tool result clearing: once a tool result lies deep in the history, the raw output is removed. Anthropic calls this one of the safest, lightest forms of compaction.
• Structured note-taking: the agent writes notes to a file outside the context, such as a to-do list or a NOTES.md, and reads them back later.
• Sub-agents: a sub-agent explores with its own clean context and returns a condensed summary, often 1,000 to 2,000 tokens.

Compression can also work on the text itself. LLMLingua (Microsoft, 2023) uses a small language model to drop the least informative tokens from a prompt; on its test sets it allowed up to 20x compression with little loss in performance. The browser agent Lumen, a small open-source project started in March 2026, uses two tiers: it first drops old screenshots, then has a language model summarise the history once 80% of the context is in use. A structured JSON state that the agent writes itself survives that compaction.

Where does it go wrong?

What is dropped is gone for the model. Anthropic warns that overly aggressive compaction can lose "subtle but critical context whose importance only becomes apparent later", and advises tuning the compaction prompt on real agent traces: first make sure it captures everything relevant, then cut what is superfluous. Compression also does not replace choosing what goes into the context in the first place. Anthropic's guiding principle is to find "the smallest possible set of high-signal tokens" for the task.

Example

In September 2025 Anthropic added context editing to its developer platform: it clears stale tool calls and results automatically as an agent approaches the token limit (announcement). Anthropic reported that in a 100-turn web search evaluation, agents with context editing finished workflows that otherwise failed on a full context, while using 84% fewer tokens. On its internal evaluation of agentic search, context editing alone improved performance by 29%, and by 39% together with a memory tool that stores information in files outside the context.