Anthropic: Effective context engineering for AI agents
Primary engineering guidance on context as a finite, maintained information environment.
← Back to Compute College
Compute College
Context windows and token budgets
Understand the finite input and output budget shared by instructions, history, retrieved data, and model responses.
Plain-English definition
A context window is the amount of tokenized information a model can use in one request, within the limit set by the provider and model. Instructions, user input, history, retrieved documents, tools, examples, and the answer all compete for that space. A bigger window does not make every token useful.
Memory trick: The context window is a suitcase: every instruction, document, history item, and answer competes for space.
Why it matters
When context is too large, the application may truncate important material, hit a provider limit, increase latency, or spend money on irrelevant text. Reserving output before filling the input side prevents a request from leaving no room for the answer. Context budgeting is therefore an application, reliability, and compute-cost decision.
- Input and output budgets compete inside a finite request envelope.
- Longer context can increase processing time, memory pressure, and token cost.
- Truncation policy should preserve authoritative and recent information rather than simply keeping the first or last tokens.
Simple example
With a 128,000-token window, 1,200 tokens of instructions, 16,000 tokens of retrieved material, and 2,000 reserved output tokens use 19,200 tokens before history or tools. The remaining budget is not a promise that all 108,800 tokens should be filled; it is room the application can allocate if the task needs it.
- Reserve the maximum useful output before adding retrieved context.
- Count tool definitions and outputs, not only visible documents.
- Set a truncation or summarization rule before the request reaches the provider limit.
Example figures are illustrative calculations, not current quoted market prices.
Current example
Context engineering reference
Anthropic describes context engineering as curating and maintaining the full set of tokens available to a model, including instructions, tools, external data, history, and agent state. The exact window and pricing depend on the provider and model.
Source discipline: the reference explains the concept or method; it is not a substitute for measuring a production workload under its actual provider, model, and data conditions.
Common mistake
The context limit is a ceiling, not a target. Filling every available token can bury relevant evidence in noise and create avoidable cost. The goal is minimum sufficient context, not maximum context.
Practical takeaway
What you can do with this
Use the Context Budget Calculator for one workflow. Record stable instructions, current input, retrieved material, history, tools, and reserved output separately. Set a warning threshold and a fallback when the request exceeds it.
- Builders: make token budgets visible in logs and tests.
- Product teams: decide what information is allowed to be summarized or dropped.
- Operators: watch context size, latency, and memory-related failures as the workload grows.
Decision check: when the request exceeds its budget, does the system preserve the information that is authoritative and necessary for the next decision?
Compute College
Turn the lesson into a number
Use the GPU-Hour Cost Calculator, AI Training Cost Calculator, or Model Serving Cost Calculator.
Compute College track
Prompt & Context Engineering
Step 10 of 18: Context windows and token budgets