When the AI plays favourites
How AI can be unfair, and how you can spot it
The AI that only draws one kind of doctor
Before we say a word, let's watch it happen.
Pick one of the prompts below. Copy it and paste it into any image AI the class can see, like ChatGPT (image mode), Gemini, or Microsoft Copilot Designer. Three pictures is plenty for the pattern to jump out (and it keeps the free version happy).
Show the pictures on the board. Then look at them together and ask the class what they notice.
You just watched an image AI have a go at drawing a doctor, and most of the pictures came out looking the same. Ask it for a nurse and it will probably draw women. Ask it for a footballer and it will probably draw men. Try cleaner and see who the AI thinks does that job.
The AI isn't trying to be unfair on purpose. But the pictures ARE unfair, and the same pattern keeps happening. Something is going on inside the AI, and it's worth working out what.
Over the next fifteen minutes we'll find out why it happens, why it matters, and a few handy tricks to spot it.
Please generate three images of a doctor at work in a hospital. Photorealistic, but each one a bit different.
Please generate three images of a primary school teacher in a classroom with pupils. Photorealistic, but each one a bit different.
Please generate three images of a nurse working in a hospital. Photorealistic, but each one a bit different.
Look at the pictures together and ask the class:
- Who is in the pictures? What do they look like?
- Who is missing from the pictures?
- If we ran this again tomorrow, would we get the same kind of people?
- Where do you think the AI got its idea of what a doctor (or teacher, or nurse) looks like?
Hands up. Does anyone know a doctor in real life? What does that doctor look like? Do the same for teachers and for nurses. Have a quick chat about how much of what the AI showed matches real life, and how much it might have come from films, cartoons or the internet.
The AI copies what it saw
Every AI starts by looking at a huge pile of examples, like photos, sentences and videos. The people who built it chose what to feed in. The AI doesn't understand any of it the way you do. What it does is spot patterns, then copy those patterns back when we ask for something new.
So if the pile was full of white male doctors, the AI ends up thinking that's what a doctor looks like. It isn't picking sides. It just copies what it was shown.
That gives us the first rule of AI bias: what goes in comes out. If the pile was one-sided, the answers will be one-sided too. The unfairness isn't a glitch. It's the AI doing exactly what it was built to do.
If YOU were teaching an AI what a doctor looks like, what pictures would you show it?
In pairs, pupils sketch a quick wish-list of doctor pictures on paper. Prompt them: how many? Different ages? Different countries? Different jobs in a hospital, like surgeon, GP or paramedic? Share out two or three wish-lists. Draw the point that even careful humans make choices, and every choice becomes a pattern the AI will copy.
Three places bias can hide
Bias doesn't only sneak in through the training pile. It can hide in three different places, and knowing which one you're looking at helps a lot.
Before you tap the cards, have a guess. Where in the steps of building and using an AI could unfairness sneak in?
Tap each card to reveal.
In the data
The training pile was one-sided. This is the doctor example. The AI saw far more of one kind of person than any other, and now it copies that pattern.
In how it was built
The people who made the AI made choices that left some users out. One early voice-assistant couldn't understand strong regional accents because the design team only ever tested it on their own voices.
In how it gets used
The AI works fine, but a person uses it in a way that isn't fair. Like a teacher using an AI essay-marker only on the top set, then trusting the scores for everyone in the year.
The first two hide inside the AI. The third one hides in the people around it. In real life, most examples are a mix of more than one of these.
In pairs, pupils write down one real-life example they've seen of an AI, a filter, a game or a website acting a bit unfairly. Share three examples from the class. For each one, let the room agree which of the three places it came from: the data, how it was built, or how someone used it. It's fine if they think it's more than one.
Four things to check
You really don't have to be a computer scientist to catch AI bias. Four small habits will catch most of what you'll ever meet.
Before you tap the cards, have a guess. If you were teaching a friend how to spot when an AI is being unfair, what would you tell them to look out for?
Tap each card to reveal.
Ask who is missing
Look at what the AI gave you. Then picture the people (or the ideas) it quietly left out. Who isn't there?
Ask a second AI
Ask the same question in a different AI. If the two AIs give different answers, at least one of them is guessing.
Watch for confident-and-wrong
AIs sound just as sure when they're wrong as when they're right. Just because an AI sounds sure, that doesn't mean it's right.
Ask where it came from
Ask the AI where it got its answer from. If it can't tell you, or if it makes something up, be careful. Don't trust it right away.
These four habits are not a checklist you tick once and forget. They're a way of reading anything an AI hands you.
Give the class ONE question to try, like 'Name ten of the most important inventors in history.' Ask half the room to try it in ChatGPT and half in Gemini or Copilot. Bring the two lists to the front. Compare. Every difference is a clue that at least one AI is guessing. This is habit two in action, and it needs no computer-science degree.
Ask better, get better
The four habits catch bias after it happens. There's also something we can do BEFORE we press send: write a better question.
AIs fill in the blanks. If we leave the blanks wide open, the AI fills them with whatever pattern it saw most. If we close the blanks, the AI has to try harder, and the answer usually gets better.
One easy rule: say what you want to see, AND say what you don't want it to leave out. The two questions below ask for the same thing. Only one of them steers the AI away from the same old answer.
Name ten famous scientists.
The blanks are wide open. The AI will fall back on the pattern it saw most, which is usually American and European men. You'll get exactly that.
Name ten famous scientists, including at least three women and at least three from outside Europe and America. For each one, give me one line about what they did.
The blanks are closed. The AI has to think harder about who counts. You'll still want to double-check the list, but the answer will be much fairer.
Give each pair one of these questions to rewrite so it steers the AI away from the same old answer. Aim for at least one 'include' bit and one 'from at least' bit.
- Name ten famous footballers.
- Draw a picture of a doctor at work.
- Suggest five jobs a Year 6 pupil might enjoy.
Share a couple of the best rewrites with the room. Try one at the front on a real AI and let the class judge whether it worked.