Temperature in AI

Temperature is a setting in many AI tools that controls how much the answers vary. At a low setting the model sticks to the most likely wording, which suits a webshop that wants product descriptions in the same style every time. At a high setting it takes more chances, which helps when you want ten different slogans. The setting has no effect on what the model knows.

How does it work?

A large language model writes one token at a time. At each step it gives every possible next token a score, called a logit. The softmax function turns those scores into probabilities, and the model draws the next token from them. Temperature divides the scores before that step. Holtzman et al. (2019) write it as exp(logit / t), normalised over all tokens.

Below 1 the differences between scores grow, and the likeliest token wins more often. Above 1 they shrink, and unlikely tokens get a real chance. At 0 the model always takes the likeliest token, which is called greedy decoding. Temperature works together with top-p sampling and top-k, which cut off the unlikely tail before the draw.

Which values do providers allow?

OpenAI's API accepts 0 to 2; according to its SDK documentation, values like 0.8 make the output "more random" and values like 0.2 make it "more focused and deterministic", and you should change temperature or top_p "but not both". Anthropic's API reference allows 0 to 1, defaults to 1, and advises values "closer to 0.0 for analytical / multiple choice, and closer to 1.0 for creative and generative tasks".

Newer models narrow the choice. In September 2026 the same Anthropic reference marks temperature as deprecated: models released after Claude Opus 4.6 accept only 1.0. Microsoft's Azure OpenAI documentation lists temperature as unsupported for its reasoning models other than GPT-6 Astra.

Does temperature 0 give the same answer every time?

Not reliably. Anthropic's reference states that "even with temperature of 0.0, the results will not be fully deterministic". In September 2025, Thinking Machines Lab sampled 1,000 completions at temperature 0 from Qwen3-235B with one prompt and got 80 different texts. The main cause they found is server load: the batch size changes from request to request, and the usual inference code gives slightly different numbers for different batch sizes. With batch-invariant code, all 1,000 completions were identical.

Does a higher temperature make answers worse?

For problem solving, less than often assumed. Renze and Guven (2024) tested nine models on multiple-choice problems and found no statistically significant effect on accuracy between 0.0 and 1.0. Above 1.0, GPT-3.5's text became incoherent quickly, and its accuracy fell to about zero at 1.6. Holtzman et al. describe the other end: lowering the temperature improves quality at the cost of diversity, and always taking the likeliest token produces text that is "bland and strangely repetitive". The survey by Huang et al. (2023) adds that a higher temperature raises the risk of hallucinations.

Example

Suppose a webshop uses one model for two jobs. Product descriptions from a fixed template run at a low temperature, so the tone stays the same across the catalogue. Campaign slogans run at a higher one, so each request brings new ideas. With the settings swapped, the product texts would drift in tone and the slogans would start to look alike. With a model that only accepts the default, you steer variety through the prompt instead, for example with a strict template for the descriptions and a request for ten different options for the slogans.