← All lessons

Non-Determinism: Why AI Doesn't Always Say the Same Thing

This is a full teaching resource on non-determinism, the fact that AI can give different answers to the same question, with activities and framings across every UK Key Stage from Y1 to Y13

Filter by key stage or audienceClick one or more chips to filter the sections below.

FAQ for teachers

Common questions teachers ask when running this lesson.

Why does the AI give a different answer every time?

That is the whole lesson. Every word an AI writes is picked from a probability distribution. It is essentially rolling weighted dice at every step. Two runs of the same prompt take two different paths through those dice rolls. This is called non-determinism, and it is on purpose. The opening 'Run It Twice' section makes it visible in ten minutes with any class.

Does turning down the AI's 'temperature' setting get rid of non-determinism?

It reduces it substantially but not always fully. Temperature controls how much the AI weighs less-likely next-token choices. At zero, the model picks the top-scoring token every time, so two identical prompts usually produce identical answers. Yet many deployed chatbots add small randomness at other layers, or route your prompt to different underlying servers, so true zero-variance behaviour is not guaranteed.

Are all AI systems non-deterministic, or only large language models?

Only some. Traditional programs, a calculator or a payroll system, are deterministic by design: same input, same output, every time. Large language models and image generators sit at the other end because they sample from probability distributions. Recommender systems, spam filters and search rankers sit in between, mostly deterministic but with randomised elements added to keep results fresh. Non-determinism is a choice, not an accident.

What if pupils get very different results from the AI demo?

That is the lesson. Ask them to compare answers, note the differences, and discuss why the outputs diverged. The variety is the point, not a fault to fix. If a pupil says 'the AI is broken', that is exactly the moment to introduce non-determinism as the answer. The classroom noise around that comparison is often the most memorable part of the hour.

Does non-determinism mean AI cannot be trusted for maths, code, or anything with a right answer?

It means the same prompt can produce different answers, some correct and some wrong. The workaround is verification, not avoidance. Run the prompt twice; if the maths matches, trust it more; if it disagrees, dig in. For code, run the code. For citations, check the source. Non-determinism is a reason to build a checking step into the workflow, not to avoid AI outright.

Can two identical prompts ever produce identical answers?

Yes, but only in narrow conditions. If the temperature is set to zero, the same server is used, no context or memory has changed, and the model has not been silently updated, then the outputs can match exactly. Change any one of those variables and the answers diverge again. Most consumer chatbots do not expose those controls, so identical replies are rare in classroom use.

Is non-determinism a bug or a feature?

A feature, and a deliberate one. A model that produced the same reply every time would be repetitive, brittle, and often stuck on a mediocre answer that happened to score highest. Sampling from a range of plausible next tokens produces the variety, creativity and paraphrase we actually want from a chatbot. The cost is unpredictability, which we then manage with prompting, verification and temperature control.

How does non-determinism affect using AI for medical or legal work?

It raises the stakes on verification. A radiologist asking a chatbot to summarise a scan cannot trust that two runs of the same query produce the same summary, so any AI-derived clinical detail must be cross-checked against the primary record. Legal drafting is the same: a clause paraphrased by AI in one session may reappear differently in another. Non-determinism forces a human-in-the-loop discipline.

Common misconceptions

What pupils tend to think, and what to say back.

Pupils often say

The AI is broken because it keeps saying different things.

It's actually

It is working exactly as designed. Every word is chosen from a range of likely next words, so a second run picks a different route through the same landscape. Consistency is the exception in these systems, not the norm.

Try asking

Would you call a die broken because it landed on a different number the second time?

Pupils often say

There must be a right answer if I ask it enough times.

It's actually

Repetition reveals the model's favourite answer, not the true one. A chatbot that has never seen the correct information will confidently give you its most-likely wrong answer over and over. Popularity inside the model is not evidence outside it.

Try asking

If a hundred people in one village all said the same thing, would that make it true?

Pupils often say

If I ask five times and count, the majority answer must be correct.

It's actually

The majority answer tells you what the model finds most probable, given what it was trained on. That is a fact about the training data, not about the world. Majority voting across runs sharpens the model's bias, it does not remove it.

Try asking

What would you need to add to the count for the majority answer to actually mean something?

Pupils often say

AI temperature is like actual heat inside the computer.

It's actually

It is a borrowed word, not a physical measurement. Temperature is a number that controls how adventurous the model is when picking each next word. Low temperature means it plays safe with the most likely word, high temperature means it takes bigger risks.

Try asking

If the AI has no thermometer, what is the temperature actually measuring?

Pupils often say

Chatbots use randomness because the developers were lazy.

It's actually

The randomness is deliberate. A chatbot that always gave the single most likely next word would sound flat, repetitive, and often plain wrong. A little variety lets it write fluently, explore alternatives, and handle open questions where there is no one right reply.

Try asking

What would a chatbot with zero randomness be useful for, and what would it be terrible at?

Pupils often say

If I press regenerate, the new answer is closer to the truth.

It's actually

Regenerating just draws a new sample from the same probabilities. The second answer is not more accurate, it is just different. If the first answer was wrong for a factual reason, the tenth will often be wrong in the same way.

Try asking

What would you check before deciding a new answer was actually better than the last one?

Pupils often say

Setting the temperature to zero makes it deterministic and accurate.

It's actually

Temperature zero makes it more predictable, not more accurate. The model will pick its top-ranked word every time, but if its top-ranked word is wrong, it will now be wrong the same way every time. Predictable and correct are different things.

Try asking

Would you rather have a friend who is confidently wrong every time or occasionally right?

Pupils often say

If it is random then it is basically just guessing.

It's actually

It is weighted randomness, not blind guessing. Every next word carries a probability shaped by billions of training examples, so common patterns come out often and nonsense almost never. It is closer to a heavily loaded dice than a blank one.

Try asking

What is the difference between rolling a fair dice and rolling one that is weighted?

Pupils often say

The AI is literally rolling dice inside the computer.

It's actually

It is using a pseudo-random number generator, which is a mathematical formula that produces numbers that look random but are actually fully reproducible if you know the starting seed. Real randomness is not required for the chatbot to feel unpredictable.

Try asking

If two computers used the same seed, would their answers agree?

Pupils often say

A calculator always gives the same answer, so a computer should too.

It's actually

A calculator runs a fixed rule on fixed inputs, so the output is locked. A chatbot samples a next word from a probability distribution, so the output is a draw, not a calculation. They are different kinds of tool sharing the same box.

Try asking

Which sort of question would you rather trust to a calculator, and which to a chatbot?

5-minute prep

Five ready-to-run ways to open this lesson. Pick one, copy the prompt, paste it into ChatGPT or Copilot.

Starter5-8 min

Ask the same question three times

The single most powerful non-determinism demo. Paste the identical prompt into three fresh chats and let the class see the answers refuse to line up.

I would like to show a class that AI does not give the same answer twice. Please write a four-sentence story about a fox who visits the moon. Do not add any preamble, warnings, or explanation. Just the four sentences. In a moment I will open a brand new chat window and paste this exact same message in again, and then a third time, so the class can compare the three answers side by side. Please answer naturally each time and do not try to keep any earlier version consistent.

Compare8-10 min

Two chatbots, one question

Run the same prompt in two different chatbots side by side. What agrees is baked into the underlying data. What differs is the randomness at work.

Please give me a short paragraph, no more than fifty words, describing what happens inside a black hole. Do not add any caveats, disclaimers, or extra text before or after the paragraph. In a moment I will paste this exact prompt into a different chatbot and put the two paragraphs next to each other on a classroom screen. The class will look at what both chatbots agreed on and what they said differently, so please answer as naturally and freely as you would normally.

Explain10-12 min

Why does the answer keep changing?

Get the AI to explain, in classroom English, why its own answers vary. Perfect for pivoting the class from noticing the variability to understanding the mechanism.

Imagine you are talking to a curious class who have just watched you give three different answers to the exact same prompt. Please explain, in three short paragraphs of plain classroom English, why this happens. Cover: (1) how you pick each word from a range of likely next words rather than a single fixed one, (2) what the 'temperature' setting does to that range, (3) why developers deliberately leave some randomness in rather than forcing you to give the same answer every time. Do not apologise for the variability, just explain it.

Check12-15 min

Measure how much the answer wobbles

Turn variability into a proper measurement exercise. Ten runs, tally the outcomes, and calculate exactly how unstable the answer is.

I am running a lesson that turns AI variability into a data-collection exercise. Please answer the following question ten times in a row, numbered one to ten, with a fresh answer each time: 'In one sentence, what is the most important invention of the last two hundred years?' Then, underneath the ten answers, produce a small table that groups the ten sentences by which invention was named, gives a count for each, and calculates the percentage of the ten runs that picked the most-common invention. Do not smooth or diversify the answers on purpose.

Repair12-18 min

Design a reproducibility protocol

The advanced move. Ask the AI to design a proper protocol for making its own answers reproducible, then have the class judge whether the protocol would actually hold up.

The class has just measured how much your answers vary across ten runs of the same question. I would now like you to write a short reproducibility protocol that a teacher or researcher could follow to make your answers as consistent as possible when they ask the same prompt again next week. Cover: (1) the specific settings to change and why, (2) exactly how to phrase the prompt to reduce variability, (3) the checks to run afterwards to confirm the answers actually agree, and (4) two things the protocol still cannot guarantee, and why.