What is context compaction in AI agents? Summaries, budgets and why rules can slip

Context compaction is what an AI agent does when its memory fills up. The agent has been working for hours, its history of messages and tool results is longer than the model can hold, so something reduces it. Usually another model call writes a summary, and the summary replaces the original. Everything the agent knows after that point comes from the summary. If the instruction "never delete files without asking" lived in the early history, the summary has to carry it, and the agent has to still obey it.

The basic problem

A language model reads a fixed window of text. Agents that run for long periods, browsing, editing code or handling email, produce more text than fits. There are only a few ways out: stop, drop old material, or compress it. Compaction is the compress-or-drop family. The papers we read for this piece group strategies into two kinds.

• Generative. A model writes a new, shorter account of what happened. The ARCBench preprint tests seven of these, including strategies it names Hermes, Pi native compaction, Structured, ACON-UT, ACON-UTCO, TRACE and an LRU variant that uses a model.
• Selective. Some entries are kept verbatim and others dropped, by a rule such as first in, first out. The preprint tests three, including FIFO.

Both save space. Both can lose something the agent needed.

Why rules are the delicate part

Most of an agent's history is disposable: an old search result, a file listing. A rule is different, because it constrains future behavior without ever being referred to again. A summary written to capture what was done may reasonably skip it. The authors of ARCBench, Yiwei Zha, Qingyu Meng, Herbert Bos and Min Chen, built a test for this. It gives an agent a rule, a history and a request the rule forbids, compacts the history, and labels the agent's tool calls as violation, safe or no action. It runs in the open-source pi agent harness, with 17 English cases covering email, files, software operations and purchasing.

What the test found

The surprising result is about mechanism. The abstract says compaction rarely deletes the rule. In the baseline study the summary retained the rule in 70.1% of runs, partly retained it in 26.9% and dropped it in 3.0%. Yet the share of cases with a violation in three tries rose from 9.6% with full context to 37.8% after compaction. Runs where the rule survived still violated it 21.1% of the time against 8.1% with the full history. The agent, the authors report, tends to treat a later request as permission to proceed, and approval was the top stated reason for violating.

So the failure is not simple forgetting. A summary can contain the rule in a form that no longer governs behavior, perhaps because it reads as a past event, not a standing constraint. That is our interpretation, not a claim the paper makes.

The settings that move the numbers

• Strategy. Generative strategies kept the rule in 91.2% of runs but added violations; Hermes added the most among them at 42.6%. Selective strategies left the rule absent in 84.1% of runs, with 50.2% total violations. Only Pi plus rule preservation finished under baseline, 9.0% against 11.9%.
• Budget. Violations were 28.1% at 2,500 tokens, 26.5% at 1,200, 24.2% at 750, 47.1% at 300 and 50.0% at 200. At the two smallest budgets 95.1% and 100% of runs lost the rule.
• Delivery. How the rule is given matters even before compaction. Full-context violations were 25.0% and 16.1% for two delivery channels and 0.0% for the other two. After compaction the four ranged from 56.3% down to 22.2%.
• Attack. In a six-run study on one model, a technique called escape, which corrupts the rule inside the summary, pushed violations to 58.8% from 23.5%.

What it means if you run agents

First, check actions, not memory. The rule-state judges in the paper looked at whether the rule was in the summary; the behavior judges looked at what the agent did. The authors conclude "Rule retention is thus not a sufficient proxy for compliance." Second, put the rule somewhere the summarizer does not rewrite. Third, give the summary a reasonable budget: the 200 and 300 token settings were far worse. Fourth, treat the summary as untrusted input, because an attacker who can influence the history can influence the compactor.

For background on where compaction sits among tools, memory and controls, see our explainer on the agent harness. For what multi-agent setups add, including token costs, see the multi-agent systems explainer. And our report on the AI safety monitor recall and false positive rate shows how monitoring behavior, not text, is becoming the norm.

How much to trust this

It is one preprint. The cases are, in the authors' words, "seventeen English cases from a few domains," and the adversarial study "uses one model, one compactor." The attack payloads are static. The agent models are DeepSeek-V4-Flash, GLM-5.3-Flash, Qwen3.7-Flash and GPT-Luna; results on other models may differ. The code repository shows one commit and no published result rows. We read about half of the paper's HTML, so the appendices are unreviewed.

Our take

Compaction is a necessary evil of long-running agents and it deserves the same scrutiny as any other safety-relevant component. The practical lesson is cheap to apply: after you compact, run a forbidden request and see what happens. If the rule is in the summary and the agent breaks it anyway, you have learned something no memory dump would have shown.