Learn AI compute, then follow the market
← Back to Compute College

Compute College

Define the AI task and success criteria

Turn a vague AI request into a measurable task with a clear outcome, constraints, and acceptance criteria.

Plain-English definition

Defining an AI task means stating what the system must do, who the result is for, what information it may use, what constraints apply, and how a reviewer will decide whether the result is acceptable. Success criteria turn a request such as “summarize this report” into an observable outcome: for example, identify the three material findings, cite the supporting passages, omit unsupported claims, and return the result in a specified format.

Memory trick: A prompt tells the model what to do. Success criteria tell the team whether it did it.

Why it matters

A model cannot reliably optimize for a success condition that the product team has not defined. Clear task boundaries reduce prompt revisions, ambiguous outputs, retries, and human correction. They also make compute economics visible: a task that needs fewer retries, shorter outputs, or a smaller model can reduce tokens, latency, and cost per successful result.

  • A measurable task gives evaluators something specific to grade instead of asking whether an answer merely feels good.
  • Acceptance criteria reveal whether a cheaper or smaller model can complete the work reliably enough.
  • Clear boundaries prevent extra context, unnecessary tool calls, and long responses from becoming recurring workload cost.

Simple example

Replace “Review this support ticket” with “Classify the ticket as billing, access, bug, or other; identify the customer’s requested outcome; cite the sentence that supports the classification; and return valid JSON with `category`, `requestedOutcome`, and `evidence`. A successful result uses one allowed category, includes evidence from the ticket, and leaves a field empty rather than inventing information.

  • The task names the decision: classify the ticket and extract the requested outcome.
  • The evidence requirement makes unsupported confidence visible during review.
  • The output contract gives software a predictable shape to validate.

Example figures are illustrative calculations, not current quoted market prices.

Current example

Primary references on task definition

Prompt-engineering guidance consistently treats clear instructions, relevant context, desired format, and examples as inputs that improve model behavior. These references support the design principles here; they do not prove that a particular prompt will work for every workload.

OpenAI prompt-engineering best practices

Guidance on stating the task clearly, supplying context, specifying format, and using examples.

Source discipline: provider guidance describes useful techniques, but teams should test their own representative tasks and record the model, prompt version, and evaluation conditions.

Common mistake

“Make this better” is a request, not an evaluation. It may start a useful conversation, but it is not enough to compare prompts or models. Define better for whom, according to which criteria, with what unacceptable outcomes and what evidence of success.

Practical takeaway

What you can do with this

Write a one-page task contract before refining a prompt: goal, input, output, constraints, failure cases, acceptance criteria, and the smallest useful test set. Ask a second person to grade sample outputs using only that contract; disagreements reveal missing definitions.

  • Product teams: write acceptance criteria that reflect the user outcome, not just model wording.
  • Developers: separate deterministic validation from model judgment wherever a rule can be checked in code.
  • Procurement and finance teams: estimate cost per accepted result, including retries and review time.

Decision check: could two reviewers use the written criteria to agree whether the same output passes? If not, the task is not defined enough to optimize.

Compute College track

Prompt & Context Engineering

Step 2 of 18: Define AI task success criteria