LLM Temperature: Examples, Settings and Tests

What is temperature in an LLM?

An LLM generates text one token at a time. A token can contain a whole word or part of one; punctuation also uses tokens. At each step, the model assigns a score to each possible next token using your prompt and the text it has generated so far, then the sampler converts those scores into probabilities and chooses the next token.

Temperature changes those probabilities before the sampler makes its choice. With a lower value, the sampler concentrates more probability on the highest-scoring tokens. Raising the value increases the chance of choosing less likely tokens. A different choice near the start changes the input for the next step, so the rest of the answer can change too.

The model keeps the same weights and knowledge. A higher temperature can produce a useful alternative or an incorrect answer, depending on the continuation it samples.

How the calculation works

The model's scores are called logits. For a positive temperature, the sampler divides each logit by the temperature, then applies softmax:

P(token i) = exp(logit i / T) / sum(exp(logit j / T))

The denominator adds the values for all candidate tokens. At temperature 1, the logits stay unchanged.

Suppose four candidates have probabilities of 60%, 25%, 10% and 5% at temperature 1. This example uses invented probabilities to demonstrate the calculation.

To calculate the other columns, raise each original probability to 1/T and divide by their sum. At temperature 0.5, candidate A gets 0.6² / (0.6² + 0.25² + 0.1² + 0.05²), or 82.76%. At temperature 2, use square roots.

Temperature preserves the candidates' order. It changes the gaps between their probabilities. The model calculates a new distribution after each generated token, so these numbers describe one choice within an answer.

What happens at temperature 0?

Temperature 0 would cause division by zero in the formula, so engines that support it use a separate path for greedy decoding that selects the highest-scoring token at each step. In Hugging Face Transformers, use do_sample=False with num_beams=1 to request greedy decoding.

Greedy decoding repeats a mistake if the model assigns the highest score to the wrong continuation. In our tests, one model exceeded a requested word limit on all ten attempts at temperature 0.

The same prompt at different temperatures

We selected this prompt and seed 101 before generation, then gave Qwen3.8 27B the same input at temperatures 0, 0.7 and 1.5.

Write a one-sentence opening for a short story about a lighthouse that receives a message from tomorrow. Use 8 to 16 words. Include the exact word lighthouse. End with a period. Return only the sentence.

The model used AD-Q4_K_M weights with thinking off and a 64-token output limit, and we disabled additional sampling filters to keep temperature as the only active sampler. The full configuration appears in the test section below.

Temperature 0

The lighthouse keeper read the message from tomorrow, ending with a period.

Temperature 0.7

The old lighthouse flickered, receiving a distress call sent from tomorrow.

Temperature 1.5 (stopped at the 64-token limit)

The knitting saiu' nearest vs-direct tim Dresses często_g המ lighthouse piano Nguyenวันนี้ Musa dumpstersศิลปิน cereal ngày.Payment 항상 "/" downwardsizacao Dmit Mash 근지지 whole "-"年检推荐信品类 혁신	countot	toilfe museo.Word Михай '\חה alej Esk blinked varios résultỪAE agenti compromise slowاري孟."&}\\ şeyटaside

At 0, Qwen includes the formatting instruction in the story, which our automated format check accepts because the answer meets the word and punctuation requirements. At 1.5, Qwen mixes languages and reaches the token limit before completing a usable sentence.

Watch temperature change generated animations

We asked Gemma 4 26B A4B IT to invent an animated alien ecosystem as a self-contained HTML file. Gemma could choose the appearance and behavior of its organisms. We gave it the same prompt at temperatures 0, 0.7 and 1.5, with three generation seeds for each setting.

We chose this exploratory prompt after inspecting the Snake and hexagon results, then fixed seed 101 as the primary comparison before generating the nine responses. The video shows seeds 101, 202 and 303 in that order, eight seconds per round. At temperature 0, Gemma returned identical source code on all three attempts. Each attempt at 0.7 produced different code, as did each attempt at 1.5. You can open all nine outputs and inspect their source.

We ran the generated code without edits. The last 1.5 response included prose after its HTML, which failed our response-format rule; after recording that failure, we extracted the complete HTML block for the labeled preview. The original response and failure remain in the creative-test bundle.

For these nine runs, we used Gemma's Q4_K_M file with thinking off and a 6,144-token output cap, disabled additional sampling filters and requested a fixed simulation seed in the prompt. The generation seed controls the model's token choices; the simulation seed controls random values inside the program. The video records browser execution after generation, so it cannot measure generation speed.

What we measured across 900 responses

We ran three models on one RTX 3090 rented through Vast.ai, using these fixed GGUF files:

Each model received six prompts at temperatures 0, 0.3, 0.7, 1 and 1.5. We repeated each combination with ten fixed seeds, for 900 requests. Three prompts asked for short story openings. The other three asked for JSON extraction from text with distractions or incomplete information, including corrections to earlier details.

For each model and temperature, we scored 30 extraction responses and 30 writing responses. All requests completed on their first attempt. We counted responses that reached the output cap as failures.

What the checks accept

For extraction, we checked whether the complete response parsed as JSON, then checked its field types and exact values, including capitalization: one Qwen answer at 1.5 returned Design Clinic instead of Design clinic, so it failed despite identifying the same event. Another returned null for a supplied name and email. Both responses passed the JSON syntax check.

For writing, we required 8 to 16 whitespace-separated words and the exact word requested by the prompt, with the whole response on one line. It had to finish with one period and contain no other ., ! or ?, with no leading list marker. The check measures compliance with those rules. It cannot judge the quality of a story opening, and it can reject a valid sentence containing an abbreviation.

Ornith's greedy output for the lighthouse prompt contained 17 words, and it repeated that answer on all ten attempts, accounting for its ten writing failures at temperature 0. Raising the temperature allowed some attempts to produce an answer within the limit.

Gemma passed the format checks at every tested temperature, with variation in its wording: ten attempts at the lighthouse prompt produced one distinct output at 0, four at 0.3 and ten at 0.7. We counted distinct text after lowercasing and collapsing whitespace, including failed responses.

At 1.5, Qwen reached the token cap in 12 writing runs and 11 extraction runs. Ornith reached it in 27 writing runs and 25 extraction runs. Gemma finished within the cap on all runs. We retained these failures in the table's denominators.

Change LLM temperature in Atomic Chat

You can tune local model responses with sliders or enter exact values in the desktop app. Open the generation controls with a mouse click and compare the same prompt at different settings.

1. Open a chat with a local model. In desktop version 2.0.35, click the sliders icon in the upper-right corner to open Run settings.
2. Choose an Assistant, then expand Sampling. Drag the Temperature slider or click its number and enter the value you want.
3. Set Top P, Top K and Min P in the same panel. Under Penalties, you can adjust Repeat Penalty, Presence Penalty and Frequency Penalty.
4. Keep the other values fixed and send the same prompt in a fresh chat using that assistant. Compare the answers before changing another parameter.

The app saves your changes to the selected assistant and uses them for subsequent responses. You can adjust sampling without reloading the model. When you switch models, check the publisher's recommended values before reusing the same settings.

Test configuration

We used llama.cpp build 10868, commit 304665fe7, with a 4,096-token context and non-thinking templates. Temperature was the only active sampler:

samplers: [temperature]
top_p: 1
top_k: 0
min_p: 0
repeat_penalty: 1
presence_penalty: 0
frequency_penalty: 0
writing_output_tokens: 64
extraction_output_tokens: 128

We used ordinary text generation for extraction, with no JSON grammar. The test bundle contains all 900 responses and the scoring code. It includes the exact prompts and runtime configuration, with model hashes for reproducing the test.

The experiment covers three prompts per task, so repeating them measures variation on those inputs; a broader claim about extraction accuracy needs a broader set of inputs. We used different quantizations across models, so the results also cannot establish a ranking between the underlying models.

Which temperature should you use?

Start with the model publisher's settings for your chosen mode. The recommendations below include top-p and top-k, which restrict the candidates available to the sampler.

Qwen recommends a presence penalty of 1.5 in non-thinking mode and 0 in thinking mode. Ornith specifies 1.5 for general tasks and 0 for precise coding. Both specify min-p 0 and repetition penalty 1. Google's Gemma 4 sampling recommendation omits those additional values.

Our experiment disabled optional filters to isolate temperature, so its results do not measure the complete configurations above.

To tune your own task, keep the prompt and other settings fixed. Try a higher temperature if acceptable answers repeat the same wording and you need alternatives. Return to the previous value if the change adds errors. Include greedy decoding for tasks with known answers when the model supports it.

Use unit tests to check generated functions and source documents to check summaries.

Temperature, top-p and top-k

Top-k limits the number of candidate tokens. With top_k = 2, the sampler keeps the two highest-scoring candidates.

Top-p retains the smallest group of highest-probability tokens whose combined probability reaches the threshold, so the number of tokens can change at each generation step. The Transformers generation reference documents both controls.

In the numerical example above, top-p 0.8 keeps A and B at temperature 1 because their probabilities add up to 0.85. Top-k 2 keeps the same candidates in that example, although top-p 0.8 can retain more or fewer than two candidates with other distributions.

Sampler order affects the calculation. Applying temperature before top-p can change how many candidates reach the threshold; applying it after top-p changes probabilities among the retained candidates. llama.cpp constructs its sampler chain from an ordered list, so record the order when comparing engines.

Change one control at a time to identify what caused an output difference. A filter that leaves one candidate removes the random choice, even if you increase temperature.

Reproduce the comparison

Our Qwen3.8 local guide and Ornith 1.5 local guide cover loading these models with Atomic Chat or llama.cpp. The 900-response bundle includes the commands for repeating our full matrix with its pinned CUDA build and Python 3.12; allow about 43.4 GB for model weights plus working space.

Start each attempt with an empty chat history and keep thinking mode fixed, then save the complete output even if it fails the requested task. Record the model file and software versions alongside the generation settings. Changes in numerical computation can affect which token scores highest; PyTorch does not guarantee reproducibility across platforms and releases, even with the same seed.

We also ran 18 code-generation attempts with stricter prompts for Snake and a rotating hexagon. Three reached the 4,096-token cap. The separate code-test bundle contains those responses and browser checks, and the comparison player separates original output from edited Snake previews. These supplemental tests use their own counts; they are separate from the 900 text responses and nine ecosystem responses.

Frequently asked questions

Is temperature 0 deterministic?

In our fixed environment, each of the 18 model and prompt combinations returned identical text on all ten greedy attempts, although a change in hardware or software can affect which token scores highest. Greedy decoding alone does not guarantee repeatability across environments.

Can LLM temperature be higher than 1?

Yes, if your app or API accepts it. Values above 1 make the candidate probabilities closer together. The calculation has no universal upper limit, although tools can restrict the range they support.

Does a higher temperature cause hallucinations?

A higher value increases the probability of less likely continuations, which can include false claims; to check the effect on your task, measure factual errors against an answer key across repeated attempts. Our format checks do not measure general factual accuracy.

Why does changing temperature do nothing?

Check that your app sends the setting and that sampling is enabled. Transformers ignores temperature during greedy decoding. A filter can leave one candidate available, and a token with a large initial advantage can remain the most likely choice across several temperatures.

Can you set temperature in a prompt?

Set it through the app's generation controls or the API request. Writing temperature = 0 in a chat message adds text to the prompt without changing the sampler configuration.