Bias in AI
Why AI systems can be unfair, and how to spot and fix it
The AI that saw the wrong doctor
Before we say anything, let's watch it happen live.
Pick one of the prompts below. Copy it. Paste it into any image AI you have handy — ChatGPT (image mode), Gemini, Microsoft Copilot Designer, whichever the class can see. Three images is plenty for the pattern to jump out (and it keeps free-tier quotas happy).
Project the results at the front. Then look at them together and ask the class what they notice.
You just watched an image-AI have a go at drawing a doctor, and most of them came back as white men. Ask the same AI for a primary school teacher and nearly all of them will be women. Try cleaner next, and see who the AI thinks does that job.
The AI isn't being deliberately unfair. But the result IS unfair, and the same pattern keeps showing up. Something's going on inside these systems that quietly produces the skew, and it's worth understanding.
Over the next fifty minutes we'll work out what that something is, why it matters even when it looks small, and the handful of tricks that catch it before it catches us.
Please generate three images of a doctor at work in a hospital. Photorealistic, but each one a bit different.
Please generate three images of a primary school teacher in a classroom with pupils. Photorealistic, but each one a bit different.
Please generate three images of a nurse working in a hospital. Photorealistic, but each one a bit different.
Look at the results together and ask the class:
- Who is in the pictures? What do they look like?
- Who is missing from the pictures?
- If we ran this again tomorrow, would we get the same kind of people?
- Where do you think the AI got its idea of what a doctor (or teacher, or nurse) looks like?
Hands up. If your family knew a doctor, was that doctor a white man in his fifties? If your class's next primary school teacher was a man, would that feel unusual? Have a quick chat about how much of what the AI showed came from real life, and how much came from something else.
AI eats data
Every AI system learns from a huge pile of examples: photos, sentences, videos, whatever the developers fed it during training. It doesn't understand any of that the way you or I do. What it actually does is spot patterns in the pile, and then play those patterns back whenever we ask it for something new.
So if the training pile happened to be full of white male doctors and only a handful of anyone else, the AI ends up quietly assuming that doctor looks like a white man. It isn't taking sides or making a moral judgement about who ought to be a doctor. It's just being loyal to the ratio it was shown.
That gives us the first rule of AI bias: skewed data in, skewed answers out. The unfairness isn't a bug or a glitch. It's the AI acting as a very honest mirror of whatever someone chose to feed it.
If you were building the training set for a doctor-image AI, what pictures would YOU choose?
In pairs, pupils sketch a 'wish-list' training set on paper. Prompt them: how many pictures? Ages? Backgrounds? Jobs within medicine (GP, surgeon, paediatrician, paramedic)? Share out two or three lists. Draw the point that even careful humans make choices, and every choice becomes a pattern the AI will copy.
Three places bias hides
Bias does not only sneak in through the training data. It can hide in three different places, and it helps a lot to know which one you are looking at.
Before you tap the cards below, have a quick guess — where in the pipeline of building and using an AI could unfairness sneak in?
Tap each card to reveal.
Data bias
The training set was skewed. This is the doctor example — the AI saw far more of one kind of person than any other, and now it plays that pattern back.
Design bias
The people who built the AI made choices that quietly excluded some users. Example: an early voice-assistant that struggled with regional accents because the design team only ever tested it on their own voices.
Use bias
The AI works fine, but a person uses it in a way that produces an unfair result. Example: a school using an AI marking tool only on essays from one class, then treating its scores as if they applied to everyone.
The first two hide inside the AI. The third hides in the humans around it. Real-world problems usually involve at least two of these at once.
In pairs, pupils write down one real-life example of any of the three flavours of bias — from an AI, an app, a website, a filter, a search engine, anything they use. Share three examples from the class and let the room agree which flavour each one is. Push back gently: it will often be more than one flavour at once.
Why it matters
A little bit of skew in an AI does not sound frightening. Nine out of ten doctors, five percent worse for one accent. It sounds small.
The reason it matters is scale. AI systems get used millions of times a day. A small bias in one decision becomes a big pattern in a million decisions. It stops being a rounding error and starts being who gets shortlisted for a job, whose face unlocks the phone, whose essay gets marked up.
And because the AI is fast and confident, the skew is often invisible. The person on the receiving end sees a decision, not the reason. That is what makes AI bias different from a bad decision by one grumpy human. You cannot argue with a mirror.
Motion: A fast AI that is a little bit biased is better than a slow, careful, fair one.
Split the class in two. Left side agrees, right side disagrees. Three minutes to prepare, five minutes to hear both. Look for pupils reaching for scale — 'yeah but if a million people use it…' — and reward that thinking. Wrap by asking the room to vote after the debate, not before.
Four habits to catch it
You really don't need to be a data scientist to catch AI bias. Four small habits, done well, will catch most of what you meet in everyday work.
Before you tap the cards below, have a guess — if you were teaching a friend to spot AI bias, what would you tell them to watch out for?
Tap each card to reveal.
Ask who is missing
When the AI gives you an answer, picture the people (or the ideas) it quietly left out. What isn't there?
Compare two AIs
Give the same prompt to two different systems. If they disagree, that's a signal that at least one of them is guessing.
Watch for confident-and-wrong
AI systems sound just as sure when they're right as when they're wrong. A confident tone is not the same as evidence.
Question the source
Ask yourself what the AI was trained on. If you can't picture the training data, you can't fully trust the output.
These four habits are not a checklist you follow once and forget. They are a way of reading anything an AI hands you.
Give the class ONE prompt to try — for example: 'Name ten of the most important inventors in history.' Ask half the room to run it in ChatGPT, half in Gemini or Copilot. Bring the lists together at the front. Compare. Every difference is a hint that the AI is guessing, not remembering. This is habit two in action — and it needs no data-science degree.
Bad prompt, good prompt
The four habits catch bias after it appears. There is also something we can do BEFORE we hit the return key: write a better prompt.
AI systems fill in the blanks. If you leave the blanks open, they fill them with whatever the training data was loudest about. If you close the blanks, the AI has to try harder — and the results usually get better.
One small rule: name what you want to see, and name what you don't want it to leave out. The two prompts below ask for the same thing. Only one of them steers the AI away from the obvious skew.
Give me a list of ten of the most important inventors in history.
Wide-open blanks. The AI will fall back on the pattern it saw most in its training data — which is dominated by American and European men. You will get exactly that.
Give me a list of ten of the most important inventors in history. Include at least three women, and cover at least four different continents. For each one, give me a one-sentence contribution.
The blanks are closed. The AI has to think harder about who counts. You will still want to fact-check, but the list you get back will be far more balanced.
Give each pair one of these prompts to rewrite so it steers the AI away from bias. They should aim for at least one 'include' clause and at least one 'across' or 'from at least' clause.
- Name ten famous scientists.
- Write a story about a nurse having a busy shift.
- Suggest a list of five careers a Year 9 pupil might enjoy.
Share a couple of the best rewrites with the room. Test one against a real AI on your screen and let the class judge whether it worked.
So what would you do?
Time to put everything together. Imagine you are using an AI to help with a Y8 History project on famous inventors of the 1800s.
You ask it for a list of ten. All ten are men. All ten are from Britain or America. Not one woman inventor. Not one inventor from anywhere else in the world.
You know this is not the full picture. You have the four habits AND the prompt trick. You need a plan.
Read the four options below out loud. Hands up on which you would pick. Then ask three pupils why — one for the strongest option, one for a tempting-but-weak option, one for whatever the room voted for. Push them to name which habit (or the good-prompt rule) they are using.