The same prompt can answer differently by design

Start here

Ask an assistant the same question twice and the wording, or even the conclusion, can differ both times, with no error and no update to the model in between. This is not inconsistency for its own sake; it is sampling. Language models produce a probability for each possible next token rather than one certain answer, and a setting called temperature controls how often the reply picks a lower-probability option instead of the safest one. OpenAI's chat API reference states this directly: higher values like 0.8 make output more random, while lower values like 0.2 make it more focused and deterministic.

What the documents say

The three vendors document the same idea with different defaults and different fates for the setting. Anthropic's Messages API reference gives temperature a 0.0-to-1.0 range with a default of 1.0, recommends values near 0.0 for analytical tasks and near 1.0 for creative ones, and states that models after Claude Opus 4.6 no longer accept a non-default temperature, rejecting the request with an error. OpenAI's reference documents a 0-to-2 range and an alternative called top_p, or nucleus sampling, restricting the model to the smallest set of next-token options whose combined probability crosses a threshold; it advises changing temperature or top_p, not both. Google's text generation documentation shows the same setting passed through a generation_config parameter, without the tuning detail the other two provide. None documents a value guaranteeing an identical reply to an identical prompt.

Check this

A reader who needs a reproducible answer, such as an extracted number, should not treat one response as authoritative just because it looks confident; sending the same request several times and comparing outputs is a direct check the mechanism invites. Anyone building a workflow around an API can look up whether their model still accepts a temperature setting at all, since Anthropic's documentation shows the parameter being retired on its newest models rather than merely tuned.

What holds and what fails

The claim that identical prompts can yield different, equally plausible outputs holds as a description of how these systems generate text at all, and it fails only for tasks that do not require precision, where the variation is harmless. It is an editorial judgment, not something the documents say outright, that a reader should treat a single generated answer the way they would treat an unverified estimate: plausible, not confirmed. Where a vendor has deprecated the setting, as Anthropic has for its newest models, adjusting temperature is no longer an available lever, and prompting technique has to do that work instead.

• Send an important factual question more than once and compare the answers before trusting one.
• Check whether temperature is still a supported parameter on the exact model version in use.
• Lower temperature, where available, for tasks with one correct answer rather than a creative one.

Variation between identical requests is not a defect to debug; it is a documented setting with a name, a range, and on at least one vendor's newest models, a scheduled retirement. Reading the parameter list is the fastest way to stop mistaking a sampling choice for a mistake.