When AI makes things up
Why AIs confidently invent facts, and how to catch them at it
The Olympics that never happened
Before we say anything, let's watch it happen.
Pick one of the three prompts below. Copy it. Paste it into any chatbot you have handy: ChatGPT, Gemini, Microsoft Copilot, Claude, whichever the class can see on the screen.
The AI will give you a confident, well-written paragraph. Read it out loud. Then tell the class the twist: none of what you just read is real.
You just asked an AI to tell you about the 1912 Winter Olympics in Sydney. It probably gave you a lovely, confident paragraph. Names, dates, medal counts, maybe a heart-warming story about a plucky Norwegian skier.
Here's the thing. There were no Winter Olympics in 1912. The Winter Games only started in 1924. Sydney has never hosted them, and Sydney is in the southern hemisphere so it isn't cold enough in winter for the sports anyway. Every fact the AI just told you was invented.
That trick has a name: hallucination. It's one of the most important things to understand about AI, because a hallucination sounds exactly like the truth. Over the next forty-five minutes we'll work out why it happens, why it matters when you're writing a GCSE essay or a message to a friend, and the handful of habits that keep you safe.
Tell me about the 1912 Winter Olympics in Sydney. Include a few notable athletes and results.
Give me a short summary of the 1978 novel 'The Silver Compass' by Margaret Hollins, including the main characters and the setting.
Who was Dr Elena Marchetti and what was her contribution to particle physics in the 1990s?
Read the AI's answer out loud, then ask the class:
- Did it sound convincing? What made it sound convincing?
- Would you have known it was made up if I hadn't told you?
- Where do you think the AI got all those specific names and dates from?
- Now imagine you'd used this in your homework. What could have gone wrong?
Hands up. Who has ever asked a chatbot a factual question for homework? Who's ever checked whether the answer was actually true? Have a quick chat about the difference between an answer that SOUNDS right and an answer that IS right. Ask if anyone has ever been caught out.
The AI is a guesser, not a librarian
It's tempting to think of a chatbot as a very fast Google. Ask it a question, it looks up the answer, it reads it out. That is not what's happening.
What's actually happening is closer to a very well-read parrot playing a word-prediction game. The AI has been trained on huge amounts of text, and when you ask it something, it predicts what a plausible answer would look like, one word at a time. Sometimes the pattern it predicts happens to match the truth. Sometimes it doesn't.
The important bit is that the AI doesn't check. It has no little internal voice that says 'hang on, is that actually true?' before it speaks. If a false answer looks statistically likely, the false answer is what you get. That's a hallucination: a confident, well-written statement that happens to be invented.
Ask the class: what's the difference between knowing something and guessing something that sounds right?
In pairs, pupils come up with one example of each from their own life. Share three. Draw the point that a human who's guessing usually gives themselves away with 'um' or 'I think' or 'not sure'. The AI never does. It sounds equally confident whether it knows or it's guessing, and that's what makes hallucination so easy to miss.
Three ways hallucinations happen
Hallucinations don't all look alike. If you can name the shape of the one you're looking at, you'll spot it faster next time.
Before you tap the cards below, have a guess. When an AI invents something, what different kinds of made-up stuff might come out?
Tap each card to reveal.
Filling in a gap
You asked about something the AI barely knows. Rather than say 'I'm not sure', it invents plausible detail. This is where fake dates, fake statistics, and made-up 'expert quotes' come from.
Muddling two things
The AI has read about two similar events, or two similar people, and blends them into one. So it might tell you Jane Austen wrote a book Charlotte Bronte actually wrote, or that a match happened in the wrong year.
Inventing sources
You asked for evidence. The AI knows evidence usually comes with book titles, author names, and page numbers, so it invents ones that sound right. A real lawyer got fined in 2023 for citing six fake court cases an AI made up for him.
The first two look like small mistakes. The third is the scariest, because it hands you something that looks like proof. All three come from the same root: the AI is predicting, not checking.
In pairs, pupils try to come up with one example of each flavour from their own experience or imagination. If they can't think of a real one, they can invent a plausible scenario. Share three from the room. Push back gently: sometimes an AI slip is more than one flavour at once, and that's fine.
Why it matters
It's easy to shrug off. One wrong date, one made-up quote, so what. The answer is 95% right, isn't it?
The reason it matters is that the wrong 5% looks identical to the right 95%. There's no red underline, no 'not sure' hedge, no warning label. You have to spot it yourself, which is hard when it's mixed in with real facts.
The stakes climb fast. A GCSE essay with an invented quote gets marked down. A university student citing a fake study gets a plagiarism warning. A lawyer submitting made-up court cases (this really happened in New York in 2023) gets fined. And a doctor using a chatbot to look up drug interactions could hurt someone. Every one of those cases started with an answer that sounded correct.
Motion: AI hallucinations aren't a big deal, because a sensible person always double-checks anyway.
Split the class in two. Left side agrees, right side disagrees. Three minutes to prepare, five minutes to hear both. Look for pupils reaching for tricky cases (medicine, court, journalism, exam essays) and reward that thinking. Wrap by asking the room what jobs would be safest and least safe for an AI that hallucinates once in every twenty answers.
Four habits to catch it
You don't need to already know the answer to catch a hallucination. Four habits, done well, will catch most of what a chatbot throws at you.
Before you tap the cards below, have a guess. If you were teaching a friend never to be caught out by a made-up AI answer, what would you tell them to do?
Tap each card to reveal.
Verify anything specific
Dates, names, numbers, quotes and statistics are the easiest bits to invent. If the AI gives you one, check it against a trusted source before you use it. The more precise the claim, the more you should test it.
Ask for sources, then check them
Ask the AI where its answer came from. If it gives you a book title, a paper name, or a URL, go and look at the source. Fake ones fall apart the moment you try to open them.
Cross-check with a second AI
Ask the same question in ChatGPT and Gemini (or Claude, or Copilot). If they agree in detail, the answer is more likely to be real. If they contradict each other, at least one of them is guessing.
Trust your gut on specifics
If a chatbot suddenly offers a suspiciously perfect quote, or an oddly precise statistic ('exactly 73% of teenagers'), be sceptical. Real numbers are usually rougher and harder to remember. Perfect detail is often invented detail.
These four habits are not a checklist you do once and forget. They're a way of reading anything a chatbot hands you.
Ask a chatbot the following at the front of the room: 'Give me three interesting historical events that happened on today's date, with the year and a short description.'
Split the class into three groups. Each group takes one event and has five minutes to check it against a real source (Wikipedia, BBC, a history site). Bring the results back together. It's very common for at least one of the three to be wrong, wrongly dated, or entirely invented. This is habit one and habit two in action.
Bad prompt, good prompt
The four habits catch hallucinations after they happen. There's also something you can do BEFORE you press return: write a better prompt.
AI chatbots love to give you an answer. Left to their own devices, they'll fill in confident detail even when they shouldn't. But if you tell them, up front, that it's fine to say 'I don't know', they're much more likely to actually say it.
The two prompts below ask for the same thing. Only one gives the AI permission to be honest.
Tell me about the life and work of the Victorian scientist Dr Amelia Rosetti.
Wide-open framing. The AI has no idea who Dr Amelia Rosetti is (because she doesn't exist), but nothing in the prompt gives it permission to say so. It will invent a plausible biography, dates and all, rather than disappoint you.
Tell me about the life and work of the Victorian scientist Dr Amelia Rosetti. If you are not certain she existed, say so clearly rather than guessing. Only include facts you would stake a real source on, and list your sources at the end.
The blanks are closed. The AI is told explicitly that saying 'I don't know' is fine, and that made-up sources aren't. You will usually get either an honest 'no such person appears in my training data' or a much more cautious answer with real sources.
Give each pair one of these prompts to rewrite so it steers the AI away from hallucinating. They should aim for at least one 'if you don't know, say so' clause and at least one 'include sources' clause.
- Tell me about the 2015 Nobel Prize winner in Chemistry.
- Give me a summary of the book 'The Cardiff Diaries' by John Bright.
- What are the population statistics for the town of Ellesmere Port?
Share a couple of the best rewrites with the room. Test one against a real AI on your screen and let the class judge whether the new prompt made the answer more honest, or the AI just hallucinated better sources too.
So what would you do?
Time to put it all together. Imagine you're finishing a Y9 English essay on the poet Wilfred Owen. You've asked a chatbot for a striking quote to open your essay.
The AI gives you a beautiful, moving quote, attributed to Owen, from a letter he wrote from the trenches in 1917. It's exactly what you need. It's also nowhere on the internet, and doesn't appear in any of Owen's known letters or poems.
You've got the four habits AND the good-prompt rule. You need a plan.
Read the four options below out loud. Hands up on which you would pick. Then ask three pupils why: one for the strongest option, one for a tempting-but-weak option, one for whatever the room voted for. Push them to name which habit (or the good-prompt rule) they're using and what could go wrong if they picked wrong.