Anthropic: Effective context engineering for AI agents
Primary guidance on context compaction, caching, and keeping long-running agent work useful.
← Back to Compute College
Compute College
Context compression, caching, and reuse
Reduce repeated context work with summaries, cached prefixes, and reusable state while preserving important evidence.
Plain-English definition
Context compression removes or summarizes information that is no longer needed while preserving decisions, evidence, constraints, and unresolved work. Caching reuses stable instructions or repeated context where the provider or application supports it. Reuse can reduce repeated processing, but a cached or summarized item must still have a freshness, scope, and invalidation rule.
Memory trick: Compression saves space. Caching saves repetition. Invalidation protects truth.
Why it matters
Many AI applications resend the same system instructions, policy excerpts, history, or tool descriptions on every request. At scale, those repeated tokens create cost and latency. Compression and caching can improve throughput, but an incorrect summary or stale cache can cause a systematic error across many requests.
- Compression trades token savings for summarization risk and an additional processing step.
- Caching helps only when the same context recurs and the provider’s accounting and invalidation behavior support it.
- Reuse should preserve source identity, effective date, permissions, and the ability to reconstruct evidence.
Simple example
An agent summarizes a completed research phase into decisions, citations, open questions, and rejected paths. Stable tool instructions are reused for a session, while current prices are retrieved again for each decision. If the policy or permissions change, the cached material is invalidated rather than silently reused.
- Compress completed work, not unresolved evidence that still needs inspection.
- Cache stable text separately from user-specific or time-sensitive context.
- Compare the summary against the original on questions that depend on caveats.
Example figures are illustrative calculations, not current quoted market prices.
Current example
Context optimization reference
Anthropic’s context-engineering guidance discusses compaction, summarization, caching, and maintaining useful agent context. The economic impact depends on provider pricing and the application’s repetition pattern, so measure actual cache hits and output quality.
Source discipline: the reference explains the concept or method; it is not a substitute for measuring a production workload under its actual provider, model, and data conditions.
Common mistake
Shorter context is not useful if compression removes the evidence or uncertainty needed for the next decision. A concise wrong memory is still wrong.
Practical takeaway
What you can do with this
Choose one repeated context block and model three options: resend, cache, or summarize. Record token volume, cache hit rate, summarization cost, latency, and answer quality. Add an explicit invalidation event for every fact that can change.
- Builders: include source IDs and timestamps in summaries.
- Operators: monitor cache behavior rather than assuming provider savings.
- Product teams: define what a user can inspect or correct after compression.
Decision check: can the system tell whether a context item is current, cached, summarized, and still authorized for this request?
Compute College
Turn the lesson into a number
Use the GPU-Hour Cost Calculator, AI Training Cost Calculator, or Model Serving Cost Calculator.
Compute College track
Prompt & Context Engineering
Step 17 of 18: Context compression caching and reuse