A reasoning model is the same kind of language model as a standard one, with one added behaviour: it spends tokens “thinking” before it answers. That sounds free. It is not. Reasoning tokens are billed as output tokens, they eat latency, and they help a lot on some tasks and not at all on others. The right question is not “is the reasoning model smarter” - it is “does this task reward extra thinking, and am I willing to pay for it.” This guide walks through what each major provider means by “thinking,” where benchmarks show it helps, where it hurts, and how to control the effort on the four major APIs.
TL;DR
- Reasoning is a behaviour, not a species. OpenAI, Anthropic, and Google expose it as an effort setting or budget; DeepSeek-R1 is steered through prompting and sampling. The OpenAI reasoning guide says reasoning models “introduce reasoning tokens in addition to input and output tokens,” and “reasoning tokens are not visible via the API [but] still occupy space in the model’s context window and are billed as output tokens.”
- It helps when the task has a right answer to find. Math, formal code planning, multi-step agentic loops, scientific QA - the DeepSeek-R1 card reports (vendor-reported) AIME 2024 at 79.8 and MATH-500 at 97.3, against 9.3 and 74.6 for the non-reasoning GPT-4o, with generation capped at 32,768 tokens.
- It hurts when the task is open-ended or latency-bound. Casual chat, rewriting, classification, voice, and short retrieval answers do not benefit. Anthropic’s guidance is to “match the starting point to the task” and to start “near the 1,024-token minimum” for simple prompts.
- Every provider gives you a knob. OpenAI exposes
reasoning.effort(values include none, minimal, low, medium, high, xhigh, and max, depending on the model). Anthropic exposes manualbudget_tokenson Claude 4.5 and earlier, andoutput_config.efforton adaptive-thinking models from Claude 4.6 onward. Google exposesthinking_level(minimal, low, medium, high) plus a dynamic-thinking default. DeepSeek-R1’s card recommends sampling settings (temperature 0.5 to 0.7) and starting the model’s output with a<think>\nprefix. - The single biggest mistake is leaving the default on. If you do nothing, current OpenAI models such as GPT-5.5, GPT-5.6, and GPT-6.1 Sol default to medium effort, Anthropic’s adaptive thinking defaults to high effort, and most Gemini models think by default. On simple tasks that can add tokens, latency, and cost.
What “reasoning” actually means across vendors
Every major provider now ships the same idea under different names. The mechanic is the same: the model generates internal tokens before its visible answer, and those internal tokens are charged as output.
OpenAI’s reasoning guide is explicit about the mechanic: reasoning models “use internal reasoning tokens before producing a response,” which helps them “plan, use tools effectively, inspect alternatives, recover from ambiguity, and solve harder multi-step tasks.” The guide recommends starting with gpt-6-astra, with gpt-5.6-terra and gpt-5.6-luna as lower-cost options; it also covers gpt-5.5 (default medium effort) and GPT-6.1 Sol. Reasoning tokens “are not visible via the API” but “still occupy space in the model’s context window and are billed as output tokens.”
Anthropic calls it extended thinking. On Claude 4.5 and earlier you set thinking: {type: "enabled", budget_tokens: N} directly. The Anthropic page calls out a hard floor: “Minimum of 1,024 tokens. The API rejects smaller values.” The budget is a target, not a cap - “Claude may stop reasoning well before the budget is exhausted; max_tokens remains the hard ceiling on total output.” On Claude 4.6 and later the manual budget is deprecated in favour of thinking: {type: "adaptive"} plus output_config: {effort: ...}. Claude 4.7 and later models, including Claude Opus 5.5, reject manual thinking entirely: they “reject requests that use it, returning a 400 error.” If you see that error today, your model uses adaptive thinking and you switch by removing budget_tokens.
Google’s Gemini thinking docs call the same behaviour “thinking” and expose it through thinking_level: "low" | "medium" | "high" (with "minimal" on some Flash models). The docs’ model table shows thinking on by default for Gemini 2.5 and 3.x models (default level medium on most Flash models, high on the Pro previews); gemini-2.5-flash-lite is the exception and ships with thinking off. The guide is firm about one trap: “max_output_tokens…sets the maximum number of tokens a response can generate, including thought tokens,” so a low cap silently cuts the thinking budget. “If the model hits this limit while reasoning, it stops generating with status “incomplete” and returns truncated or empty output (while still billing for any thinking tokens generated). To reduce cost or latency without truncating responses, lower thinking_level (low or medium) instead of setting a small max_output_tokens.”
DeepSeek exposes the same mechanic through open weights. The DeepSeek-R1 card describes a 671B-parameter Mixture-of-Experts model with 37B activated, 128K context, MIT licence, and a recommended temperature of 0.6 (range 0.5 to 0.7). The card notes that the R1 models “tend to bypass thinking pattern” on some queries, and, “To ensure that the model engages in thorough reasoning”, recommends forcing the model to begin its response with <think>\n.
The open-weight world ships the same idea with a different dial. The gpt-oss-120b model card sets reasoning effort through the system prompt: “Reasoning: low” for “fast responses for general dialogue,” “Reasoning: medium” for “balanced speed and detail,” “Reasoning: high” for “deep and detailed analysis.” It is a 117B MoE with 5.1B active, MXFP4 quantisation so it runs on a single 80 GB GPU, and an Apache 2.0 licence.
The single sentence that ties this all together: reasoning is the same behaviour with five different control surfaces, and each can be turned down, by API setting or by prompt.
Where reasoning helps
The DeepSeek-R1 card is the clearest public side-by-side of reasoning and non-reasoning models on the same benchmarks. The figures are vendor-reported by DeepSeek, with a 32,768-token generation cap and 64 responses per query for pass@1. Claude 3.5 Sonnet (October 2024), GPT-4o (May 2024), and DeepSeek V3 are standard models; o1-1217 and R1 are reasoning models:
| Benchmark | Claude 3.5 Sonnet | GPT-4o | DeepSeek V3 | OpenAI o1-1217 | DeepSeek-R1 |
|---|---|---|---|---|---|
| MMLU (Pass@1) | 88.3 | 87.2 | 88.5 | 91.8 | 90.8 |
| GPQA Diamond | 65.0 | 49.9 | 59.1 | 75.7 | 71.5 |
| LiveCodeBench | 33.8 | 34.2 | - | 63.4 | 65.9 |
| SWE-bench Verified | 50.8 | 38.8 | 42.0 | 48.9 | 49.2 |
| AIME 2024 | 16.0 | 9.3 | 39.2 | 79.2 | 79.8 |
| MATH-500 | 78.3 | 74.6 | 90.2 | 96.4 | 97.3 |
| Codeforces rating | 717 | 759 | 1134 | 2061 | 2029 |
The DeepSeek-R1 distilled checkpoints drop a lot of that performance but keep the same shape: the 32B Qwen distill still scores 72.6 on AIME 2024 and 94.3 on MATH-500. The pattern is clear in the table: the reasoning models are far ahead on competition maths and competitive programming (AIME 2024 79.8 for R1 against 9.3 to 39.2 for the standard models), a few points ahead on broad knowledge (MMLU 90.8 against 87.2 to 88.5), and not ahead at all on SWE-bench Verified, where the standard Claude 3.5 Sonnet scores 50.8 against R1’s 49.2.
The Anthropic guidance lands at the same conclusion from the practitioner side. For complex tasks, Anthropic recommends starting “with a larger budget of 16,000 tokens or more” and tuning from there: “Larger budgets can improve response quality by enabling more thorough analysis for complex problems.” For simple tasks, the recommendation is the opposite: “For simple tasks, start near the 1,024-token minimum and increase incrementally.”
Gemini’s docs are even more pointed. The recommended mapping is:
- Simple tasks: “Use minimal or low thinking for fact retrieval or classification (e.g., “Where was DeepMind founded?”).”
- Moderate tasks: “Use default thinking for comparing concepts or creative reasoning (e.g., Compare electric and hybrid cars).”
- Complex tasks: “Use maximum thinking for advanced coding, math, or multi-step planning (e.g., Solve AIME math problems).”
OpenAI’s effort table maps the same idea onto its effort values. The high-effort row is “hard reasoning, complex debugging, deep planning, and high-value tasks where quality and intelligence matters more than latency.” The xhigh row extends it: “deep research, asynchronous workflows and agentic tasks that require long runs.” The none row exists for “latency-critical tasks that do not benefit from any reasoning or multi-chained tool calls” - voice, fast info retrieval, classification. gpt-6-astra does not accept none and returns HTTP 400 if you try.
The rule that falls out of all four documents: reasoning helps when the task has a verifiable answer, a multi-step plan, or a long horizon. It does not help when the answer is “rewrite this paragraph” or “what is the capital of France.”
Where reasoning hurts
Three costs that always show up, regardless of vendor:
-
Token cost. OpenAI is explicit: reasoning tokens “are billed as output tokens.” On Anthropic, changing the thinking configuration between turns invalidates prompt caching. Anthropic’s example shows the cost: switching the budget from 4,000 to 8,000 tokens on the third request “re-creates the cache (
cache_creation_input_tokens=1370,cache_read_input_tokens=0).” If you cache a long system prompt for cost reasons, leaving thinking effort on a moving dial can quietly defeat the cache. -
Latency. Gemini’s docs warn that lowering
thinking_levelis how you cut latency without losing the visible answer. Anthropic’s warning is stronger: “Pushing the model to think beyond 32k tokens produces long-running requests that can hit system timeouts and open-connection limits.” For a budget above 32K, “use batch processing to avoid networking issues.” OpenAI “recommends reserving at least 25,000 tokens for reasoning and outputs when you start experimenting” - and the failure mode is that the response can returnstatus: incompletewithreason: max_output_tokens“before any visible output tokens are produced.” -
Quality regressions on the wrong task. Extra thinking buys nothing on a simple prompt, and the DeepSeek-R1 card flags a failure in the other direction: the model “tend[s] to bypass thinking pattern…when responding to certain queries, which can adversely affect the model’s performance.” The same model can both over-think and under-think, depending on how you prompt it. Anthropic’s tuning advice is to start small and grow: “start near the 1,024-token minimum” for simple prompts, then “increase incrementally.”
A practical pattern: keep reasoning off for classification, rewriting, retrieval, voice, and short chat. Keep it on for math, multi-step agentic planning, hard debugging, and long-horizon research. For mixed workloads, use the lowest effort that solves the hardest task in the set.
How each provider lets you control reasoning effort
The four vendors expose the same dial under different names.
OpenAI uses reasoning.effort on the Responses API (recommended for reasoning models). Supported values are model-dependent and can include none, minimal, low, medium, high, xhigh, and max. gpt-5.5 defaults to medium; gpt-6-astra does not support none; GPT-6.1 Sol supports neither none nor minimal; gpt-5.6 and gpt-6 also expose reasoning.mode: "standard" | "pro", where pro “performs more model work than standard mode, increasing token usage and cost.” For long conversations, reasoning.context keeps prior reasoning in scope: current_turn (the default on most models) only carries the active turn; all_turns (default on gpt-5.6) renders earlier reasoning as input on later turns.
Anthropic uses budget_tokens on Claude 4.5 and earlier, and output_config.effort on adaptive-thinking models from Claude 4.6. The minimum is 1,024 tokens, and budget_tokens must be less than max_tokens (except in interleaved-thinking mode). On Claude 4.6 and later, manual mode is deprecated; on Claude 4.7 and Claude Opus 5.5, manual mode is rejected with HTTP 400. The migration is one-line: remove budget_tokens, set thinking: {type: "adaptive"}, and control depth with output_config: {effort: ...} (the API default is "high"). Anthropic notes one behaviour change to expect: “With a fixed budget, Claude thinks on every request. With adaptive thinking, Claude decides whether and how much to think on each request, and at lower effort settings it may skip thinking entirely on easy inputs.” That is a feature if your workload is mixed, and a footgun if you depend on reasoning happening.
Google Gemini uses a thinking_level setting. The default is dynamic thinking, where “Gemini models engage in dynamic thinking by default, automatically adjusting the amount of reasoning effort based on the complexity of the request.” You override the level explicitly with "minimal" | "low" | "medium" | "high". The trap is max_output_tokens: it is a hard ceiling that includes thought tokens, so a low cap cuts reasoning silently. Gemini’s fix is to lower thinking_level instead.
DeepSeek-R1 and the open-weight world do not have a vendor API parameter. You control reasoning through sampling: temperature 0.5 to 0.7 (the card recommends 0.6), no system prompt, and forcing the response to begin with <think>\n to enforce the trace. Open-weight reasoning models like gpt-oss-120b take a different approach and put reasoning effort in the system prompt: “Reasoning: low | medium | high.” The card offers full access to the chain of thought for debugging, adding: “It’s not intended to be shown to end users.”
A decision framework
Five questions, in order. Stop at the first “no” or “depends, but the default works” and you have your answer.
- Does the task have a verifiable answer, a multi-step plan, or a long horizon? If yes, a reasoning model is on the table. If no, a standard model is cheaper and faster.
- Is latency or cost the binding constraint? If yes, default to the lowest effort that solves the simplest task in your workload, and turn reasoning on only for the prompts that need it. OpenAI’s
noneand Anthropic’s lower effort settings are the levers. - Do you cache long system prompts? If yes, hold the thinking configuration stable for the life of the cache. Anthropic’s example shows a single budget change re-creating a 1,370-token cache. OpenAI’s
configuration_updatebetween responses preserves the prefix - that is the safe pattern. - Is the workload mixed (simple and hard in the same batch)? If yes, prefer adaptive thinking on Anthropic, dynamic thinking on Gemini, or route per-request on OpenAI. Fixed high effort on a mixed workload overpays.
- Are you running open weights? If yes, for DeepSeek-R1 set temperature to 0.6 and start the response with
<think>\nwhen you need the trace, and accept that effort is a prompt-time choice, not an API flag.
The Bottom Line
Reasoning is a behaviour, and it is the right tool for a narrow band of tasks. Maths, competitive programming, and scientific QA show the strongest benchmark gains in DeepSeek’s own comparison (R1 at 79.8 on AIME 2024 and 97.3 on MATH-500, against 9.3 to 39.2 and 74.6 to 90.2 for standard models), while real-repository coding on SWE-bench Verified showed no gain over Claude 3.5 Sonnet. OpenAI’s effort table names hard reasoning, debugging, and long-horizon agentic work at the high end of the dial. For everything else, reasoning tokens are an output-token surcharge that does not improve the answer. Use the lowest effort that solves the hardest task in your workload, hold the configuration stable if you cache, and on DeepSeek-R1, start the response with <think>\n and use temperature 0.6 rather than reaching for a non-existent API flag.