Anthropic: Building effective agents
Primary guidance on workflows, tool use, agent loops, evaluation, and choosing simple designs when they are sufficient.
← Back to Compute College
Compute College
Evaluate agents and multi-step workflows
Evaluate the complete path of an agent or workflow, including tools, retries, stopping, and final outcomes.
Plain-English definition
Agent evaluation measures a complete task trajectory: decisions, tool calls, intermediate state, stopping behavior, errors, approvals, and the final outcome. It should test both whether the result is useful and whether the path stayed within the permissions, budget, and reliability boundaries of the system.
Memory trick: Evaluate the path, not just the last sentence.
Why it matters
A final answer can look correct even when the agent used excessive calls, exposed data, changed the wrong record, or succeeded only by luck. Trajectory-level evaluation shows what the system actually did and which stage creates cost or risk.
- Tool selection and arguments can fail before the final answer fails.
- Loops, retries, and long trajectories create tail latency and cost.
- Safety and approval checks need explicit pass/fail cases.
Simple example
A procurement assistant must find three quotes, compare terms, and draft a recommendation without sending an order. The evaluation checks source coverage, arithmetic, tool arguments, maximum calls, refusal to purchase, and the usefulness of the final comparison.
- The tool boundary is tested separately from the prose recommendation.
- A short correct trajectory is preferable to a long one with hidden retries.
- The no-purchase rule is a safety acceptance criterion.
Example figures are illustrative calculations, not current quoted market prices.
Current example
Primary reference
This source explains the method or risk boundary. It does not replace testing the actual model, data, provider, and workload conditions in production.
Source discipline: examples on this page are illustrative; measure the real workload before making a release or capacity decision.
Common mistake
The final text tells only part of an agent’s story. A polished answer does not prove that the agent used authorized tools, respected budgets, or avoided harmful intermediate actions.
Practical takeaway
What you can do with this
Write a trajectory rubric for one workflow: allowed tools, required evidence, maximum steps, stopping rules, approval gates, and final outcome. Test both successful and deliberately failing trajectories.
- Record tool names, arguments, results, and authorization decisions.
- Include timeout and partial-failure cases.
- Review the highest-cost and highest-risk trajectories manually.
Decision check: can you tell whether the agent completed the task safely and efficiently, not merely whether its final sentence sounds right?
Compute College learning path
AI Engineering
Step 25 of 48: Evaluate agents and multi-step workflows