Terminal-Bench
Official benchmark site and methodology entry point.
← Back to Compute College
Compute College
What is Terminal-Bench?
Learn what Terminal-Bench measures and why terminal-based AI agent benchmarks matter for token usage, latency, and AI compute demand.
Plain-English definition
Terminal-Bench is a benchmark for AI agents completing practical tasks in terminal environments, where systems must use tools and produce verifiable end-to-end outcomes rather than answer one prompt.
Memory trick: Terminal benchmarks test agents doing work, not just models answering questions.
Why it matters
Terminal-agent workflows may involve many model calls, commands, observations, retries, and verifications. That pattern can consume materially more inference capacity than a short chat response.
- Terminal-agent runs chain many model calls, commands, observations, and retries.
- That pattern can consume far more inference capacity than a single chat response.
- So a terminal-task gain implies a heavier, longer-running serving profile.
Simple example
A terminal task might ask an agent to build software, alter files, run tests, or process data in a controlled environment, with the final state checked for completion.
- A task might build software, edit files, run tests, then have its final state checked.
- The score reflects end-to-end completion, not a single generation.
- Runtime, tool calls, and retries drive the real cost behind the result.
Example figures are illustrative calculations, not current quoted market prices.
Current example
Primary source
The official Terminal-Bench site describes a collection of terminal-environment benchmarks for measuring agent task resolution and publishes the model, agent, token, and cost context alongside its leaderboard. Last checked: Sep 16, 2026.
Current leaderboard scores are intentionally not reproduced on this educational page.
Common mistake
Do not treat a terminal-agent result as interchangeable with a simple question-answering or single-generation score.
Practical takeaway
What you can do with this
Compare terminal agents using completion rate, runtime, model and tool calls, token spend, retries, and total cost per completed task.
- Compare terminal agents on completion rate, runtime, model and tool calls, tokens, and retries.
- Budget for the full run, not a single response.
- Validate cost per completed task in your own environment before scaling.
Decision check: have you costed the full terminal run — calls, tools, retries, runtime — rather than a single generation?
Compute College
Follow model releases as market signals
Follow model releases as AI compute market signals in the ComputeTape Market Brief.
Compute College track
Model Benchmarks & AI Compute Economics
Step 13 of 25: What is terminal bench