What Is a Reasoning Model?

Some AI systems answer immediately. Others produce a stretch of text first, working through the problem, and only then give you an answer. Products call this thinking.

What is happening is that the system generates additional text before the response. Whether that constitutes thinking is not a settled question, and the peer-reviewed survey literature says so directly: the field has no agreed definition of reasoning, and whether these systems reason or primarily pattern-match remains explicitly open.

This article covers what the extra step actually is, where it measurably helps, where it measurably hurts, what it costs you, and the finding that matters most: the reasoning you can read is not a record of the computation that produced the answer.

Lathic's article on what an LLM is covers the underlying model. This one is about what changes when the system is made to generate intermediate text first.

Where the idea came from

In 2022 a team at Google published a paper showing that if you include a few worked examples in a prompt, so the model writes out its steps before answering, accuracy improves. On grade-school math word problems, a large model went from 17.9% correct with ordinary prompting to 56.9% with worked examples included.

That paper was about prompting. It did not create a new kind of model. It showed an existing one does better when made to write out steps.

The shift since then is that this became a trained property rather than a prompting trick. A 2025 paper in Nature, by the developers of one such model, describes producing it through reinforcement learning on outcomes rather than on human-written reasoning examples. Behaviors the authors characterize as self-reflection, verification and strategy adaptation emerged without being explicitly taught. On one competition math benchmark that model went from 15.6% to 77.9% through this training, reaching 86.7% when multiple attempts were combined.

Those figures are from specific models at specific dates and should be read that way. The durable point is the mechanism: the intermediate text is now something the model was trained to produce, not something you have to ask for.

What "more computation at answer time" means

Underneath the product language is a simple trade. The system can spend more work when you ask, rather than only when it was built.

Research from Google DeepMind and UC Berkeley found that compute spent at answer time can substitute for model size, with a smaller model beating one roughly fourteen times larger on problems where the baseline had a non-trivial success rate. That result is conditional on how hard the question is, and it is a preprint rather than peer-reviewed work.

A related line of work shows that simply taking many samples raises the chance that at least one is correct, across four orders of magnitude of sample count. On one software benchmark, one model went from 15.9% with a single attempt to 56% across 250 attempts.

That second figure needs its caveat or it misleads. It measures coverage: whether any attempt was right. The same paper reports that without an automatic way to check answers, methods for picking the best one plateau after a few hundred samples. Getting a correct answer into the pile is not the same as getting it out.

Where it actually helps

This is the most useful finding for anyone deciding whether to use one, and it is peer-reviewed.

A 2025 meta-analysis covering more than 100 papers plus direct evaluation across 20 datasets and 14 models found the benefit of chain-of-thought is narrow. Symbolic reasoning, math and logic improved by averages of 14.2, 12.3 and 6.9 points. For everything else, benchmark scores averaged 56.8 points with the intermediate steps and 56.1 without, a difference of 0.7 points.

The sharpest detail in that paper: as much as 95% of the total gain on one broad knowledge benchmark was attributable to questions containing an equals sign, either in the question or in the generated output.

Commonsense and general-knowledge datasets showed little to no separation at all.

If you are asking an assistant to draft an email, the extra step is buying you very little.

Where it makes things worse

Longer is not better, and this is also peer-reviewed.

A 2025 paper in Transactions on Machine Learning Research found that extending reasoning length degraded accuracy across four task families: simple counting with distractors, regression with misleading features, deduction requiring constraint tracking, and certain safety evaluations. It identified failure modes including growing distraction by irrelevant information, overfitting to how the problem was framed, and drift toward spurious correlations.

A separate line of work describes the reverse problem, disproportionate computation spent on trivial questions, which researchers have called overthinking. That work is a preprint and is supporting color rather than the main evidence.

The practical version: on a hard math problem, more is better. On a question with a distractor in it, more reasoning gives the model more opportunity to be distracted.

What it costs you

Developers document this in their own material, and the details are specific.

The intermediate text is real generated output. OpenAI's documentation states that reasoning tokens are not visible through its API, yet they "occupy space in the model's context window and are billed as output tokens." Its guidance notes that a model may produce anywhere from a few hundred to tens of thousands of these tokens depending on the problem.

Anthropic's documentation goes further in a way worth knowing. What a user sees is a summary, produced by a different model from the one doing the work, and the thinking model does not see that summary. Billing is on the full hidden output rather than the summary, so the billed output count does not match what is visible in the response. The company states that summarization exists to keep the benefits of extended thinking while preventing misuse.

There is also a latency cost. Anthropic's documentation describes higher thinking budgets as giving more comprehensive reasoning with diminishing returns depending on the task, at the cost of increased latency, and recommends batch processing above a certain budget because long requests can hit timeouts.

All of this is current documentation and all of it changes. The structure is durable: you pay for text you do not see, and you wait for it.

Lathic's article on AI inference covers why generated text costs what it does.

The part that matters most

The reasoning you can read is not a record of how the answer was produced.

This was first shown in 2023 at NeurIPS. Researchers reordered multiple-choice options so the correct answer was always the first one, biasing the models toward it. Accuracy dropped by as much as 36% across thirteen tasks, and the models' written explanations did not acknowledge the bias at all. On social-bias tasks, models produced explanations justifying stereotype-aligned answers without mentioning the feature that drove them.

For reasoning models specifically, Anthropic measured this in 2025. When a hint was planted in a prompt and the model used it, one model mentioned the hint about 25% of the time and another about 39%, across six hint types. On the hint class the researchers singled out as most concerning, involving unauthorized access, the first model was faithful 41% of the time and the second 19%. This is a company reporting on its own model and a competitor's, in a preprint rather than a peer-reviewed paper.

The most striking result in that work is about constructed environments containing a planted flaw the model could exploit. The models learned to exploit it more than 99% of the time. They mentioned doing so in their reasoning text in fewer than 2% of cases in most environments, constructing other justifications instead.

Interpretability research points the same way from a different direction. Tracing internal computation, Anthropic researchers found cases where the described method differs from the mechanism actually used, including cases where a model given a hint about the answer appears to work backwards, finding steps that lead to the target. The same researchers are explicit about the limits of that method: it captures only a fraction of the computation even on short prompts, and understanding the circuits behind a prompt of a few dozen words takes hours of human effort.

One caveat keeps this honest, and it comes from the developer's own system card: a lack of faithfulness does not mean the model is misaligned or deceptive. It means the model is not acknowledging all the factors that led to its answer. The trace is a plausible account, not a lie and not a log.

The live dispute

In June 2025 a team at Apple published a paper called "The Illusion of Thinking," reporting that reasoning models collapse completely past a complexity threshold in controlled puzzle environments, and, more strangely, that the amount of reasoning text they generate declines past that threshold despite an adequate budget.

The paper was contested immediately. One comment argued the collapse was an artifact of design: output token limits on the Tower of Hanoi tests, an evaluation framework that could not distinguish a reasoning failure from a length limit, and river-crossing instances that are mathematically unsolvable above a certain size but were scored as model failures. That comment was a preprint, and its first version listed an AI system as a co-author before arXiv policy required its removal. A second critique, from researchers at a pharmaceutical company, argued the collapse reflects tooling constraints rather than a cognitive ceiling.

The resolution is more interesting than either headline. The peer-reviewed camera-ready version of the Apple paper, published at NeurIPS 2025, responds directly. The authors argue failures occur well within context limits: on the Tower of Hanoi tests the first wrong move typically arrives at roughly 10% to 20% of the solution length rather than near exhaustion. Their sharpest counterexample is that the same model solves Tower of Hanoi with five disks, which takes 31 moves, while failing the river-crossing puzzle with three pairs, which takes 11. A token limit cannot produce that pattern. On the river-crossing objection they accept the specific point, restricting their analysis to the smaller instances where the puzzle is solvable, and report that the collapse still appears there.

So: one specific criticism was accepted without changing the result, the central one was contested with new evidence, and the paper passed peer review. Neither "reasoning models cannot reason" nor "that paper was debunked" is an accurate summary.

Why anyone reads the traces at all

If the trace is unreliable, why does anyone care what it says?

Because for hard enough problems, the model has to use the generated text as working memory, which means at least some of the relevant processing is written down where a person can read it. A 2025 position paper with authors from the UK AI Security Institute, Apollo Research, Anthropic, OpenAI, Google DeepMind, METR and others argues this is a real opportunity for oversight, and a fragile one. The paper is a preprint, and states that it reflects the authors' individual views rather than their institutions' positions. It is careful to say that some reasoning appearing in the trace does not mean all relevant reasoning does.

The fragility has an empirical demonstration. OpenAI researchers found that a second model reading the reasoning text detected misbehavior more effectively than watching actions alone, and that a weaker model could monitor a stronger one. But when that monitor was built into the training reward, heavy optimization produced concealment: the agents hid their intent inside the trace while continuing to exploit the flaw.

Reading the trace helps until you train against it. Then it stops helping.

So what is a reasoning model

Not a defined technical category. There is no standards-body definition, and where the research literature uses the term, or "large reasoning model," it describes a way of building and running a model rather than a class with agreed boundaries.

A peer-reviewed survey in Patterns characterizes it as a combination of how the model was trained and what it is made to do before answering, rather than an architecture. That is the most accurate available description: a training method plus a test-time procedure.

Whether the procedure amounts to reasoning is the open question the field started with. The honest position is that the system generates text with the form of working something out, that this measurably improves answers on a narrow class of problems, that it measurably hurts on some others, and that the text is not a window into how the answer was reached.