Non-determinism: same question, different answer

Open ChatGPT, Gemini or whichever AI you have to hand in a second tab. Copy one of the prompts below, run it three times, and line up the three answers side by side.

Some of these prompts will produce variety that's genuinely useful. Others will produce variety you should be worried about. Your job is to work out which is which.

You ask ChatGPT to define photosynthesis for a Y12 revision card. You run the same prompt again for a friend and the sentence comes back worded slightly differently. A third run gives you a third version. All three are probably right, but none of them match, which is uncomfortable when you're building a revision resource that's meant to be identical for both of you.

That's non-determinism. Large language models don't recall answers from a database; they generate them one word at a time by picking from a probability distribution. That randomness is the whole point when you're brainstorming or drafting, but it's a genuine hazard when you need a single reliable answer. The same prompt, different answers, and no warning that it's happening.

Over the next forty minutes we'll take non-determinism apart, work out where it helps and where it hurts, and finish with the prompt habits that pin the AI down on the occasions when you really need it to stay still.

A large language model doesn't retrieve a stored answer. It generates its response one token at a time, and at every token it works out a probability distribution over what the next word could plausibly be. Then it samples from that distribution. Most of the time the top choice wins, but the second or third choice can sneak in, and once one different word has been picked, the rest of the sentence unfolds from there.

That sampling is governed by a setting called temperature. High temperature means the AI takes more chances, so you get more variety and more surprise. Low temperature means the AI plays it safe, so you get less variety and a fairly predictable answer. Consumer chatbots run at a medium setting by default, which is exactly why you get similar-but-not-identical answers each run.

The upshot: sameness is the exception, not the rule. If you actually need it, you have to design your prompt around it.

You'll see this trick in more places than you might expect. Before you tap the cards below, have a guess: where in your everyday AI use might the same prompt be quietly producing different answers?

In all three, the same underlying mechanism is running: sample from a distribution, unfold from there. Understanding this changes what you can safely trust the AI for.

This is the question that actually matters in practice. Non-determinism isn't good or bad on its own; it's good for some tasks and bad for others, and the trick is knowing which one you're doing before you press return.

The rule of thumb: if the task has one correct answer, variability is a bug. If the task has many possible good answers, variability is a feature.

Ask the AI for maths, unit conversions, spelling, factual answers, code that has to compile, legal advice, medical dose calculations or exact quotations, and you want consistency. The same input should give you the same output every time, and if it doesn't then at least one of the two runs is wrong.

Ask the AI for a poem, a story opener, an essay plan, a revision analogy, a brainstorm, alternative ways to phrase a sentence or three ideas for a project, and variety is the whole reason you're using it. The point of asking is to get options you wouldn't have thought of on your own.

The dangerous cases are when a task looks creative but actually needs a single correct answer. Or when someone runs a prompt once, trusts the output, and never realises that a second run would have said something meaningfully different.

One well-known example: in 2023, a New York lawyer submitted a legal brief drafted by ChatGPT that cited entirely fictional court cases the AI had invented (a mixture of non-determinism and hallucination, and a career-ending mistake). It happens quietly in much smaller ways all the time. A revision app might tell one pupil that Henry VIII had six wives and another that he had five, because the AI's borderline sampling landed on a different token. A medical chatbot might give slightly different dosage suggestions to different users for the same query. A shop's price-lookup bot might quote £39.99 on Tuesday and £42.50 on Wednesday.

The pattern to watch for: any time an AI's answer will be trusted as authoritative, or copied straight into a document, or used to make a real decision, sameness matters, and you cannot assume you'll get it.

You can't fully switch non-determinism off in a normal chatbot, because the temperature setting is baked in and you don't get to touch it. But you can push the AI hard towards consistency by writing prompts that leave it far less room to wander.

The trick is to pin down exactly what you want the answer to look like. Name the format. Name the length. Name the level. Name the content. Every constraint you add is one less place for the AI to roll different dice, and by the time you've named enough of them, the shape of the output stops varying even if the exact wording still drifts a little.

Time to put everything together. You're a Y12 revising for an A-level Business exam. You ask ChatGPT to explain the difference between a cash flow forecast and a profit and loss statement. Two hours later, working through a past paper, you ask again. You get a slightly different definition. Neither one is obviously wrong, but they don't match: one uses the phrase 'over a period' for cash flow, the other uses it for P&L.

You've got mocks in three weeks. You need to decide which of the two is right, and you need a workflow that stops this happening again in the run-up to the exam.