Reasoning models let you buy accuracy with compute at inference time: the model works through a problem before answering, and you pay for the working. It is the most genuinely new capability lever of the last two years, and also the easiest place to spend a budget on nothing, because the returns are extremely uneven across task types.
The distinction that predicts the outcome is whether the task has a verifiable structure. Problems with constraints to satisfy, steps to compose, or a checkable answer improve substantially with more thinking. Tasks that are recall, extraction, formatting or style do not improve at all — they were never limited by deliberation.
What the budget buys
Concretely, from evaluation work on production features: multi-constraint scheduling, root-cause analysis over logs, multi-step data derivations, and code changes touching several interacting files all improved materially with an extended thinking budget. Classification, entity extraction, summarisation, tone rewriting and routine SQL generation improved by nothing measurable, while costing several times more and adding seconds of latency.
The failure mode when it does not help is not neutral, either. Longer reasoning on an under-specified task tends to produce more confident elaboration of a wrong premise. If the task is ambiguous, more thinking buys a better-argued mistake.
# Route by task class, not by user tier. Thinking is a capability, not a perk.
THINKING_BUDGET = {
'extract_fields': 0, # structure is in the schema, not the reasoning
'classify_ticket': 0,
'summarise': 0,
'diagnose_incident': 8000, # multi-hop over evidence: pays back
'plan_migration': 12000, # constraint satisfaction: pays back
}
def answer(task, prompt):
budget = THINKING_BUDGET.get(task, 0)
resp = client.messages.create(
model=MODEL,
thinking={'type': 'enabled', 'budget_tokens': budget} if budget else None,
messages=[{'role': 'user', 'content': prompt}],
)
# Track cost and accuracy per task class, or you are tuning blind.
metrics.observe(task, thinking_tokens=resp.usage.thinking_tokens,
output_tokens=resp.usage.output_tokens)
return respLatency is a product decision
Thirty seconds of thinking is unacceptable in an autocomplete and entirely fine in an overnight analysis. That means the routing decision is as much interface design as engineering: interactive paths get a small or zero budget and stream immediately; asynchronous paths — the report, the triage queue, the migration plan — get a large budget and a progress indicator, because nobody is watching a cursor blink.
- Set budgets per task class and hold them in config, so they are tunable without a deploy.
- Measure accuracy against cost per task, not in isolation. A two-point gain for four times the price is a business decision, not a technical one.
- Do not show raw reasoning traces to end users; they read as confident and are not a reliable explanation of the final answer.
- Cache aggressively around reasoning calls — the stable prefix discipline matters more, not less, when each call is expensive.
- Re-benchmark on model updates. These trade-offs move meaningfully with each generation.
More thinking does not fix an ambiguous task. It buys you a more articulate wrong answer.
The practical posture is a cascade: a fast model with no thinking handles the bulk, a confidence or validation check catches the cases it should not have answered, and those escalate to a reasoning call. That structure costs a little accuracy against always using the expensive path, and typically a fraction of the money — and unlike a flat choice of model, it degrades gracefully when traffic spikes.