Track 1:
AI Compute 101
Start here. GPUs, GPU-hours, model costs, and why compute became a market.
0 of 7 complete
Learn AI compute in plain English.
Free lessons, calculators, and explainers for understanding GPUs, model costs, prompt and context design, cloud capacity, data centers, power constraints, and emerging AI compute infrastructure. Explore 129 published lessons, follow a learning path, or use a calculator to estimate real compute costs.
A different lesson is highlighted in the Market Brief each weekday.
Sponsor slot available
Sponsor Compute College
Reserved placement for infrastructure, data-center, energy, and AI compute sponsors.
View sponsor optionsThe curriculum
9 tracks, plain English
Pick a track and work through it in order, or jump to the topic you need. Track your progress in this browser — no account required. Every lesson is free.
A suggested route
Build the mental model, then use the market
You do not need to begin with pricing. Start with the system, understand how models use it, then apply that knowledge to buying and market signals.
Featured program
Build AI systems that work outside the demo.
AI Engineering is a practical path through prompts, context, evaluation, agents, and production workload economics. All 48 lessons are published, with browser-local tools for practice.
AI Engineering Program
Five courses and 48 published lesson placements connecting application design to tokens, latency, reliability, and compute cost.
Explore AI EngineeringTrack 2:
GPU Pricing & Capacity
What H100, H200, and B200 hours cost, and how on-demand, reserved, and spot pricing differ.
0 of 8 complete
Track 3:
Model Costs
What it costs to train a model and to serve one, and why utilization drives the bill.
0 of 7 complete
Track 4:
Prompt & Context Engineering
How prompts, context, retrieval, and agent state shape model quality, token usage, latency, and inference cost.
0 of 18 complete
Track 5:
Model Benchmarks & AI Compute Economics
How benchmark scores connect to token pricing, latency, throughput, and real inference spend.
0 of 25 complete
Track 6:
Power & Data Centers
Power, cooling, networking, memory, and the physical sites where AI compute actually runs.
0 of 17 complete
Track 7:
Compute Market Structure
Neoclouds, capacity markets, and reservations: how compute supply gets priced.
0 of 8 complete
Track 8:
Compute Futures
Forward pricing for compute: futures, forward curves, and forward contracts.
0 of 5 complete
Track 9:
Buyers & Operators
How labs and teams buy, compare, and budget GPU capacity in practice.
0 of 8 complete
Compute College tracks
Explore the 9 core tracks
Follow the curriculum in order or open one topic at a time: learn the AI compute system, understand models and workloads, then use pricing, infrastructure, and market lessons to make better decisions. Calculators and emerging topics sit alongside the core tracks.
AI Compute 101
What is AI compute?
AI compute is the hardware, power, networking, and cloud access used to train and run AI models.
Why compute matters
AI capability becomes a market constraint when usable compute is scarce, costly, or slow to deliver.
What is a GPU-hour? Meaning, formula, and cost
One GPU running for one hour: the formula behind AI rental cost and cloud price comparisons.
H100 vs H200 vs B200: price, memory, and performance
How NVIDIA accelerator generations compare on workload fit, current GPU-hour pricing, memory, and availability.
What is Model Training Cost?
Model training cost is the compute expense required to teach or improve an AI model from data.
What is Frontier Model Serving Cost?
Frontier model serving cost is the estimated expense of running a leading AI model for users after training.
Why power matters for AI compute capacity
Why electricity, interconnection, and site readiness now constrain GPU deployment.
GPU Pricing & Capacity
NVIDIA H100 Price Per Hour Explained
H100 price per hour is the hourly cost of accessing one NVIDIA H100 GPU for an AI workload.
H200 Price Per Hour Explained
H200 price per hour is the hourly cost of accessing one NVIDIA H200 GPU for AI workloads.
B200 Price Per Hour Explained
B200 price per hour is the hourly cost of accessing NVIDIA B200-generation AI compute capacity.
GPU vs TPU vs Custom ASIC
GPUs, TPUs, and custom ASICs are different kinds of AI accelerator that trade flexibility for efficiency on targeted workloads.
On-Demand vs Reserved vs Spot GPU Pricing
On-demand, reserved, and spot GPU pricing are three ways buyers obtain and pay for AI compute capacity.
What are GPU rentals? AI cloud pricing and capacity signals
How rented accelerator capacity turns GPU access into observable market pricing.
What are spot prices? AI GPU spot market explained
How interruptible GPU capacity prices expose marginal AI compute supply.
What is GPU Cloud Capacity?
GPU cloud capacity is buyer-accessible accelerator supply available for AI workloads through cloud providers.
Model Costs
What is Model Training Cost?
Model training cost is the compute expense required to teach or improve an AI model from data.
What is Frontier Model Serving Cost?
Frontier model serving cost is the estimated expense of running a leading AI model for users after training.
What is Cost per Million Tokens?
Cost per million tokens is how hosted AI APIs price inference — usually with input and output tokens priced separately.
What is GPU Utilization?
GPU utilization measures how much paid accelerator capacity is actively doing useful work.
What is Model FLOPs Utilization (MFU)?
Model FLOPs Utilization (MFU) measures how much of a GPU's theoretical compute a job actually uses; goodput counts useful work delivered.
How Fast Does an H100 Depreciate?
GPU depreciation spreads an accelerator's purchase cost over its useful life, which drives the real cost of every GPU-hour.
How to Estimate Monthly AI Compute Burn
Monthly AI compute burn measures recurring spending on the capacity used to train, fine-tune, experiment, and serve models.
Prompt & Context Engineering
What is prompt engineering?
How clear instructions, examples, constraints, and output formats help an AI model complete a task.
Define the AI task and success criteria
Turn a vague AI request into a measurable task with a clear outcome, constraints, and acceptance criteria.
Prompt anatomy: instructions, context, and output
Learn the main parts of a production prompt and how each one affects model behavior, tokens, and validation.
Clear instructions and constraints
Write direct AI instructions with priorities, boundaries, edge cases, and constraints that reduce ambiguity without creating a brittle prompt.
Few-shot examples and counterexamples
Learn when examples improve AI behavior, how to choose representative cases, and how example tokens can change workload cost.
Structured outputs and output contracts
Use schemas, required fields, and deterministic validation to make AI responses usable by software.
Prompt decomposition and chaining
Break complex AI work into smaller stages and decide which steps belong in code, prompts, or human review.
Prompt versioning and regression testing
Track prompt changes, test representative cases, and detect quality or workload regressions before release.
What is context engineering?
How AI systems select and manage the information a model receives before making a decision.
Context windows and token budgets
Understand the finite input and output budget shared by instructions, history, retrieved data, and model responses.
What belongs in context?
Select the smallest set of relevant, authoritative, recent, and complete information needed for the next AI decision.
Retrieval-augmented generation explained
Understand how an AI application retrieves external information and supplies it to a model before generation.
Chunking and document preparation
Prepare documents and retrieval chunks so relevant information stays together, searchable, and traceable.
Ranking, reranking, and retrieval quality
Learn how retrieval systems order candidate evidence and how to measure whether the right material reaches context.
Long context is not the same as good context
Learn why adding more tokens can increase cost and distraction without improving an AI answer.
Conversation history, state, and memory
Separate recent messages, durable facts, task state, and retrieved knowledge in a long-running AI workflow.
Context compression, caching, and reuse
Reduce repeated context work with summaries, cached prefixes, and reusable state while preserving important evidence.
Untrusted context and prompt injection
Keep user text, retrieved documents, tool results, and memory from silently overriding the instructions that govern an AI system.
Model Benchmarks & AI Compute Economics
What are AI model benchmarks?
Learn what AI model benchmarks measure, where they mislead, and why benchmark results can become compute demand and cost signals.
How are AI model benchmarks calculated?
AI model benchmarks compare models on fixed tasks, but their scores only become useful for AI compute buyers when read with cost, latency, and token use.
Why AI model benchmarks can be misleading
Learn why AI benchmark scores can mislead buyers when they hide prompt setup, retries, tool use, latency, token usage, and model serving cost.
How to compare model quality vs cost
Learn how to compare AI model benchmark performance with token pricing, latency, throughput, and cost per useful result.
Benchmark score vs production cost
Learn why higher AI benchmark scores may not lower production cost, and how token usage, latency, retries, and context size affect serving spend.
How to estimate cost per completed AI task
Learn how to estimate the full cost of an AI task, including input tokens, output tokens, retries, tool calls, latency, and model selection.
Model latency explained
Learn what AI model latency means, why it matters for production workloads, and how it connects to model serving cost and infrastructure capacity.
Tokens per second explained
Learn what tokens per second means, how model throughput affects AI applications, and why throughput matters for AI compute capacity planning.
Context window explained
Learn what an AI model context window is and how longer context affects token cost, memory, latency, and model serving economics.
What is a coding benchmark?
Learn what AI coding benchmarks measure and why coding-agent benchmarks matter for inference demand, model serving cost, and AI compute capacity.
What is SWE-bench?
Learn what SWE-bench measures, why it matters for AI coding agents, and how software-engineering benchmarks connect to AI compute demand.
What is LiveCodeBench?
Learn what LiveCodeBench measures, why fresh coding tasks matter, and how contamination-resistant coding benchmarks affect AI model evaluation.
What is Terminal-Bench?
Learn what Terminal-Bench measures and why terminal-based AI agent benchmarks matter for token usage, latency, and AI compute demand.
GPT-6 Astra benchmark explained
Read GPT-6 Astra benchmark claims through AI compute economics: agent quality, token use, API pricing, context length, and inference demand.
Claude Opus 5 benchmark explained
Read Claude Opus 5 benchmark claims through AI compute economics: task quality, effort settings, token pricing, fast mode, and inference demand.
Historical: Claude Opus 4.8 benchmark explained
Read Claude Opus 4.8 benchmark claims as AI compute economics evidence: capability-per-dollar, effort settings, fast mode, agent workloads, and serving demand.
What is Claude Mythos Preview?
Claude Mythos Preview is an unreleased Anthropic frontier model used in Project Glasswing for defensive cybersecurity work.
What is GPQA Diamond?
Learn what GPQA Diamond measures, why expert science reasoning benchmarks matter, and how they connect to frontier AI compute demand.
What is MMLU-Pro?
Learn what MMLU-Pro measures, how it differs from older academic benchmarks, and why benchmark difficulty matters for AI model evaluation.
What is Humanity’s Last Exam?
Learn what Humanity’s Last Exam measures and why frontier academic benchmarks matter for model capability claims and AI compute demand.
What is a reasoning benchmark?
Learn what AI reasoning benchmarks measure and how reasoning scores connect to model serving cost, latency, and frontier AI compute demand.
What is an agent benchmark?
Learn what AI agent benchmarks measure and why agentic workflows can drive higher token usage, latency, retries, and AI compute demand.
How model releases affect AI compute demand
Learn how new AI model releases can change inference demand, training demand, token usage, cloud GPU capacity, and the AI compute market.
Why output tokens cost more than input tokens
Learn why output tokens usually cost more than input tokens and how generation cost affects model serving economics, AI agents, and inference spend.
Why Reasoning Models Cost More to Serve
Reasoning models generate long chains of thought before answering, multiplying output tokens — and output tokens drive inference cost.
Power & Data Centers
Why power matters for AI compute capacity
Why electricity, interconnection, and site readiness now constrain GPU deployment.
What is an AI data center? Power, cooling, and GPU capacity
The facility stack that turns accelerators into operating AI compute supply.
Why cooling matters
Cooling removes the heat created by dense AI hardware so a facility can deliver usable compute safely.
Why networking matters for AI clusters and training cost
How interconnect quality turns GPU count into useful clustered compute.
Why memory matters for AI accelerators and HBM supply
How high-bandwidth memory affects model fit, GPU value, and accelerator availability.
What is an AI Cluster?
An AI cluster is a connected system that turns many GPUs and supporting infrastructure into usable model-training or serving capacity.
What is NVLink?
NVLink is a high-speed GPU connection technology that helps accelerators coordinate work inside AI systems.
What is InfiniBand?
InfiniBand is high-performance networking used to connect servers in many large AI clusters.
What is NVL72? Scale-Up vs Scale-Out
NVL72-style rack systems link many GPUs into one; it illustrates scale-up (bigger tightly-coupled units) versus scale-out (more networked units).
What is Liquid Cooling?
Liquid cooling removes heat from dense AI hardware so more compute can operate reliably in a facility.
What is Data Center Interconnection?
Data center interconnection links AI capacity to networks, clouds, data sources, and buyers who need to use it.
PUE Meaning and Power Usage Effectiveness Formula
Power Usage Effectiveness measures how much facility electricity is required to deliver useful IT power.
Power Sourcing for AI: PPAs and Behind-the-Meter
AI data centers secure electricity through PPAs, behind-the-meter generation, and firm low-carbon sources like nuclear and SMRs.
What is a Megawatt of AI Compute?
A megawatt of AI compute is a power-based way to describe possible data-center and accelerator capacity.
What is a Data Center Interconnection Queue?
A data center interconnection queue is the waiting line for large facilities seeking electrical grid connection.
What is AI Data Center Cooling Density?
AI data center cooling density describes the heat-removal capacity required by concentrated GPU racks.
What is High-Bandwidth Memory (HBM)?
High-bandwidth memory is fast memory located near advanced accelerators to keep AI workloads supplied with data.
Compute Market Structure
What is a neocloud? Meaning, examples, and GPU capacity
How compute-first cloud operators sell GPU clusters, reservations, and AI capacity.
What is a Compute Capacity Market?
A compute capacity market organizes access to AI compute as priced, reserved, allocated, or future-delivered capacity.
What is GPU Cloud Capacity?
GPU cloud capacity is buyer-accessible accelerator supply available for AI workloads through cloud providers.
What is a Compute Reservation?
A compute reservation secures defined GPU or accelerator capacity for a buyer over an agreed period.
What are GPU rentals? AI cloud pricing and capacity signals
How rented accelerator capacity turns GPU access into observable market pricing.
What are spot prices? AI GPU spot market explained
How interruptible GPU capacity prices expose marginal AI compute supply.
What is GPU-Backed Financing?
GPU-backed financing is borrowing against GPUs or their rental contracts to fund AI infrastructure without paying the full cost upfront.
What is Sovereign AI Compute?
Sovereign AI compute is AI capacity a country controls within its own borders and jurisdiction, reducing dependence on foreign infrastructure.
Compute Futures
What are compute futures? AI GPU forward pricing explained
How forward prices turn future GPU capacity into a planning and market signal.
How to read a compute forward curve for AI GPU capacity
How curve shape signals expected tightness, relief, or uncertainty in future compute supply.
What is a Compute Forward Contract?
A compute forward contract agrees today on a price for defined compute capacity delivered at a future time.
What is a Compute Capacity Market?
A compute capacity market organizes access to AI compute as priced, reserved, allocated, or future-delivered capacity.
What is a Compute Reservation?
A compute reservation secures defined GPU or accelerator capacity for a buyer over an agreed period.
Buyers & Operators
How AI Labs Buy Compute
AI labs secure usable capacity through rentals, reservations, cloud agreements, owned clusters, and strategic infrastructure deals.
How to Compare GPU Cloud Quotes
Comparing GPU cloud quotes means normalizing rate, capacity quality, access terms, and expected completed-workload cost.
GPU Cloud Quote Comparison Checklist
Eleven concrete fields to compare on every GPU cloud quote before signing.
How to Estimate Monthly AI Compute Burn
Monthly AI compute burn measures recurring spending on the capacity used to train, fine-tune, experiment, and serve models.
When to Use Spot GPUs vs Reserved Capacity
Spot GPUs suit flexible work; reserved capacity suits predictable or critical AI workloads that need dependable access.
What is a Neocloud Service Level Agreement?
A neocloud service level agreement defines reliability and remedy terms for specialist GPU-cloud capacity.
What is AI Compute Procurement?
AI compute procurement is the disciplined process of sourcing and managing accelerator capacity for AI workloads.
What is GPU Utilization?
GPU utilization measures how much paid accelerator capacity is actively doing useful work.
Tools & Calculators
GPU-Hour Cost Calculator
Estimate AI compute cost from GPU price, runtime, utilization, and overhead.
AI Training Cost Calculator
Estimate a training-run budget using GPU-hours and operating assumptions.
Model Serving Cost Calculator
Estimate recurring inference cost from usage and capacity needs.
Reserved vs On-Demand Calculator
Compare committing to reserved GPU capacity against paying on-demand for a workload.
API vs Self-Hosted Calculator
Compare paying per token for a hosted API against running the model on your own GPUs.
More Lessons & Emerging Topics
Agent loops, stopping conditions, and budgets
Control repeated model and tool steps with explicit limits, progress checks, and cost budgets.
Agent observability and evaluation
Instrument agent trajectories so teams can explain decisions, failures, latency, and cost.
Agent security and excessive agency
Limit what an AI agent can access, decide, and execute when tools or data are involved.
AI workload patterns
Compare batch, interactive, streaming, retrieval-heavy, and agentic workload patterns before sizing compute.
Build a practical evaluation dataset
Create a small, representative evaluation set that covers normal traffic, edge cases, and important failures.
Cost per successful outcome
Measure the total workload cost required to produce an accepted AI result.
Define functional and nonfunctional requirements
Separate what an AI system must do from how well, safely, quickly, and reliably it must do it.
Define the task, ground truth, and scoring rubric
Turn an AI evaluation goal into a task definition, reference answer, and scoring rubric that reviewers can apply consistently.
Deterministic graders and rule-based checks
Use code and explicit rules for the parts of AI evaluation that do not require model judgment.
Evaluate agents and multi-step workflows
Evaluate the complete path of an agent or workflow, including tools, retries, stopping, and final outcomes.
Evaluate retrieval and grounded answers
Measure whether retrieval finds useful evidence and whether the final answer stays grounded in that evidence.
Historical: Claude Opus 4.7 benchmark explained
Read Claude Opus 4.7 benchmark claims as AI compute economics evidence: capability, token pricing, workload fit, and likely inference demand.
Hosted API vs managed deployment vs self-hosting
Compare three ways to run an AI workload by control, latency, capacity, operations, and total cost.
Human approval and reversible actions
Design approval gates and undo paths before an AI system can take consequential actions.
Human review and model-based graders
Combine human judgment and model-based grading without mistaking either one for ground truth.
Latency: TTFT, output speed, and end-to-end response time
Separate time to first token, generation speed, and total response time when evaluating AI systems.
Model selection for the workload
Choose a model by task quality, latency, token economics, reliability, and deployment constraints.
Observability and GenAI operations
Operate AI systems with metrics, traces, evaluations, feedback, and change management.
Production reliability and regression gates
Turn evaluation results into release gates, monitoring, rollback, and reliability practices for AI systems.
Reliability and operational design
Design fallbacks, timeouts, retries, and service boundaries that keep AI workloads useful during failure.
Routing, fallbacks, and model portfolios
Use task-aware model routing and approved fallbacks to balance quality, latency, resilience, and cost.
Single-agent vs multi-agent design
Compare one capable agent with multiple specialized agents and the coordination cost between them.
The agent stack: model, instructions, tools, harness, and environment
Map the components that make an AI agent work and the boundary each component owns.
Throughput, concurrency, batching, and queues
Understand how request volume, concurrency, batching, and queueing determine serving capacity.
Token, context, and cache economics
Connect tokens, context size, caching, and reuse to AI workload cost, latency, and capacity.
Tool design and function calling
Design narrow, typed tools that let a model request useful actions without hiding critical assumptions.
What are Prometheus and Hyperion? Meta AI campuses explained
The latest build status for Meta’s Ohio and Louisiana AI campuses, and what remains future capacity.
What is an AI agent?
Understand an AI agent as a model-driven system that observes context, chooses actions, and works toward a goal.
What is an AI workload?
Define an AI workload by its task, traffic, model behavior, latency, reliability, and compute requirements.
What is Colossus? SpaceXAI’s Memphis GPU cluster
What SpaceXAI reports about its operating 220,000-plus-GPU Memphis cluster and why one million remains a roadmap.
What is Project Rainier? AWS Trainium cluster explained
AWS’s operating, nearly 500,000-chip Trainium2 system and how custom silicon changes compute supply.
What is Stargate? OpenAI’s AI infrastructure buildout
How to separate Stargate sites that are operating, under construction, and still planned.
What is Terafab? SpaceX chip factory explained
SpaceX’s announced Texas semiconductor project, its first-phase commitments, and what must happen before chips reach the market.
When not to build an agent
Choose a deterministic workflow when autonomy adds cost, uncertainty, or risk without enough benefit.
Why AI applications need evaluations
Learn why representative evaluations are necessary before shipping or changing an AI application.
Workflows before agents
Use fixed, composable steps before introducing a dynamic agent loop.
Contact Compute College
Help improve Compute College
Send lesson ideas, corrections, source material, or questions about AI compute education to the editorial team.