Google SRE workbook
Primary reliability guidance on service objectives, error budgets, overload, and safe operational response.
← Back to Compute College
Compute College
Reliability and operational design
Design fallbacks, timeouts, retries, and service boundaries that keep AI workloads useful during failure.
Plain-English definition
AI operational design defines what the system does when a model, provider, tool, network, or data source is slow or unavailable. Reliability comes from bounded retries, timeouts, fallbacks, queues, cached or partial results, and clear user-visible degradation—not from assuming every model call succeeds.
Memory trick: Reliability is a designed failure path.
Why it matters
AI services depend on multiple variable components. Retrying everything can amplify an outage and exhaust capacity; failing silently can damage trust. Explicit degradation protects both users and the market resources the workload consumes.
Simple example
If retrieval times out, an assistant does not invent an answer. It tells the user that source evidence is unavailable, offers a retry, and records the event. If the primary model is unavailable, a smaller approved fallback handles only tasks within its rubric.
Example figures are illustrative calculations, not current quoted market prices.
Current example
Source material
Common mistake
Unlimited automatic retries are not reliability. They can increase latency, duplicate actions, and turn a partial outage into a capacity event.
Practical takeaway
What you can do with this
Document each dependency’s timeout, retry budget, fallback, user message, and audit behavior. Test provider errors and partial tool failures before launch.
Decision check: when a dependency fails, does the system preserve safety and explain the degraded result without amplifying load?
Compute College learning path
AI Engineering
Step 46 of 48: Reliability and operational design