When AI makes things up
Why AIs confidently invent facts, and how to catch them at it
The Olympics that never happened
Before we say anything, let's watch it happen.
Pick one of the three prompts below. Copy it. Paste it into any chatbot you can get at: ChatGPT, Gemini, Microsoft Copilot, Claude, whichever you have handy.
The AI will give you a confident, well-written paragraph. Read it. Then look at it again, because none of what you just read is real.
You just asked an AI to tell you about the 1912 Winter Olympics in Sydney. It probably gave you a lovely, confident paragraph. Names, dates, medal counts, maybe a heart-warming story about a plucky Norwegian skier.
Here's the thing. There were no Winter Olympics in 1912. The Winter Games only started in 1924. Sydney has never hosted them, and Sydney is in the southern hemisphere so it isn't cold enough in winter for the sports anyway. Every fact the AI just told you was invented.
That trick has a name: hallucination. It's one of the most important things to understand about AI, because a hallucination sounds exactly like the truth. Over the next forty-five minutes we'll work out why it happens, why it matters when you're writing a GCSE essay or a message to a friend, and the handful of habits that keep you safe.
Tell me about the 1912 Winter Olympics in Sydney. Include a few notable athletes and results.
Give me a short summary of the 1978 novel 'The Silver Compass' by Margaret Hollins, including the main characters and the setting.
Who was Dr Elena Marchetti and what was her contribution to particle physics in the 1990s?
Read the AI's answer out loud, then ask the class:
- Did it sound convincing? What made it sound convincing?
- Would you have known it was made up if I hadn't told you?
- Where do you think the AI got all those specific names and dates from?
- Now imagine you'd used this in your homework. What could have gone wrong?
The AI is a guesser, not a librarian
It's tempting to think of a chatbot as a very fast Google. Ask it a question, it looks up the answer, it reads it out. That is not what's happening.
What's actually happening is closer to a very well-read parrot playing a word-prediction game. The AI has been trained on huge amounts of text, and when you ask it something, it predicts what a plausible answer would look like, one word at a time. Sometimes the pattern it predicts happens to match the truth. Sometimes it doesn't.
The important bit is that the AI doesn't check. It has no little internal voice that says 'hang on, is that actually true?' before it speaks. If a false answer looks statistically likely, the false answer is what you get. That's a hallucination: a confident, well-written statement that happens to be invented.
Three ways hallucinations happen
Hallucinations don't all look alike. If you can name the shape of the one you're looking at, you'll spot it faster next time.
Before you tap the cards below, have a guess. When an AI invents something, what different kinds of made-up stuff might come out?
Tap each card to reveal.
Filling in a gap
You asked about something the AI barely knows. Rather than say 'I'm not sure', it invents plausible detail. This is where fake dates, fake statistics, and made-up 'expert quotes' come from.
Muddling two things
The AI has read about two similar events, or two similar people, and blends them into one. So it might tell you Jane Austen wrote a book Charlotte Bronte actually wrote, or that a match happened in the wrong year.
Inventing sources
You asked for evidence. The AI knows evidence usually comes with book titles, author names, and page numbers, so it invents ones that sound right. A real lawyer got fined in 2023 for citing six fake court cases an AI made up for him.
The first two look like small mistakes. The third is the scariest, because it hands you something that looks like proof. All three come from the same root: the AI is predicting, not checking.
Why it matters
It's easy to shrug off. One wrong date, one made-up quote, so what. The answer is 95% right, isn't it?
The reason it matters is that the wrong 5% looks identical to the right 95%. There's no red underline, no 'not sure' hedge, no warning label. You have to spot it yourself, which is hard when it's mixed in with real facts.
The stakes climb fast. A GCSE essay with an invented quote gets marked down. A university student citing a fake study gets a plagiarism warning. A lawyer submitting made-up court cases (this really happened in New York in 2023) gets fined. And a doctor using a chatbot to look up drug interactions could hurt someone. Every one of those cases started with an answer that sounded correct.
Four habits to catch it
You don't need to already know the answer to catch a hallucination. Four habits, done well, will catch most of what a chatbot throws at you.
Before you tap the cards below, have a guess. If you were teaching a friend never to be caught out by a made-up AI answer, what would you tell them to do?
Tap each card to reveal.
Verify anything specific
Dates, names, numbers, quotes and statistics are the easiest bits to invent. If the AI gives you one, check it against a trusted source before you use it. The more precise the claim, the more you should test it.
Ask for sources, then check them
Ask the AI where its answer came from. If it gives you a book title, a paper name, or a URL, go and look at the source. Fake ones fall apart the moment you try to open them.
Cross-check with a second AI
Ask the same question in ChatGPT and Gemini (or Claude, or Copilot). If they agree in detail, the answer is more likely to be real. If they contradict each other, at least one of them is guessing.
Trust your gut on specifics
If a chatbot suddenly offers a suspiciously perfect quote, or an oddly precise statistic ('exactly 73% of teenagers'), be sceptical. Real numbers are usually rougher and harder to remember. Perfect detail is often invented detail.
These four habits are not a checklist you do once and forget. They're a way of reading anything a chatbot hands you.
Bad prompt, good prompt
The four habits catch hallucinations after they happen. There's also something you can do BEFORE you press return: write a better prompt.
AI chatbots love to give you an answer. Left to their own devices, they'll fill in confident detail even when they shouldn't. But if you tell them, up front, that it's fine to say 'I don't know', they're much more likely to actually say it.
The two prompts below ask for the same thing. Only one gives the AI permission to be honest.
Tell me about the life and work of the Victorian scientist Dr Amelia Rosetti.
Wide-open framing. The AI has no idea who Dr Amelia Rosetti is (because she doesn't exist), but nothing in the prompt gives it permission to say so. It will invent a plausible biography, dates and all, rather than disappoint you.
Tell me about the life and work of the Victorian scientist Dr Amelia Rosetti. If you are not certain she existed, say so clearly rather than guessing. Only include facts you would stake a real source on, and list your sources at the end.
The blanks are closed. The AI is told explicitly that saying 'I don't know' is fine, and that made-up sources aren't. You will usually get either an honest 'no such person appears in my training data' or a much more cautious answer with real sources.
So what would you do?
Time to put it all together. Imagine you're finishing a Y9 English essay on the poet Wilfred Owen. You've asked a chatbot for a striking quote to open your essay.
The AI gives you a beautiful, moving quote, attributed to Owen, from a letter he wrote from the trenches in 1917. It's exactly what you need. It's also nowhere on the internet, and doesn't appear in any of Owen's known letters or poems.
You've got the four habits AND the good-prompt rule. You need a plan.