When the AI plays favourites
How AI can be unfair, and how you can spot it
The AI that only draws one kind of doctor
Before you read on, watch it happen.
Pick one of the prompts below. Copy it and paste it into any image AI you can use, like ChatGPT (image mode), Gemini, or Microsoft Copilot Designer. Three pictures is plenty for the pattern to jump out (and it keeps the free version happy).
Look at the pictures. What do you notice?
You just watched an image AI have a go at drawing a doctor, and most of the pictures came out looking the same. Ask it for a nurse and it will probably draw women. Ask it for a footballer and it will probably draw men. Try cleaner and see who the AI thinks does that job.
The AI isn't trying to be unfair on purpose. But the pictures ARE unfair, and the same pattern keeps happening. Something is going on inside the AI, and it's worth working out what.
Over the next fifteen minutes we'll find out why it happens, why it matters, and a few handy tricks to spot it.
Please generate three images of a doctor at work in a hospital. Photorealistic, but each one a bit different.
Please generate three images of a primary school teacher in a classroom with pupils. Photorealistic, but each one a bit different.
Please generate three images of a nurse working in a hospital. Photorealistic, but each one a bit different.
Look at the pictures together and ask the class:
- Who is in the pictures? What do they look like?
- Who is missing from the pictures?
- If we ran this again tomorrow, would we get the same kind of people?
- Where do you think the AI got its idea of what a doctor (or teacher, or nurse) looks like?
The AI copies what it saw
Every AI starts by looking at a huge pile of examples, like photos, sentences and videos. The people who built it chose what to feed in. The AI doesn't understand any of it the way you do. What it does is spot patterns, then copy those patterns back when we ask for something new.
So if the pile was full of white male doctors, the AI ends up thinking that's what a doctor looks like. It isn't picking sides. It just copies what it was shown.
That gives us the first rule of AI bias: what goes in comes out. If the pile was one-sided, the answers will be one-sided too. The unfairness isn't a glitch. It's the AI doing exactly what it was built to do.
Three places bias can hide
Bias doesn't only sneak in through the training pile. It can hide in three different places, and knowing which one you're looking at helps a lot.
Before you tap the cards, have a guess. Where in the steps of building and using an AI could unfairness sneak in?
Tap each card to reveal.
In the data
The training pile was one-sided. This is the doctor example. The AI saw far more of one kind of person than any other, and now it copies that pattern.
In how it was built
The people who made the AI made choices that left some users out. One early voice-assistant couldn't understand strong regional accents because the design team only ever tested it on their own voices.
In how it gets used
The AI works fine, but a person uses it in a way that isn't fair. Like a teacher using an AI essay-marker only on the top set, then trusting the scores for everyone in the year.
The first two hide inside the AI. The third one hides in the people around it. In real life, most examples are a mix of more than one of these.
Four things to check
You really don't have to be a computer scientist to catch AI bias. Four small habits will catch most of what you'll ever meet.
Before you tap the cards, have a guess. If you were teaching a friend how to spot when an AI is being unfair, what would you tell them to look out for?
Tap each card to reveal.
Ask who is missing
Look at what the AI gave you. Then picture the people (or the ideas) it quietly left out. Who isn't there?
Ask a second AI
Ask the same question in a different AI. If the two AIs give different answers, at least one of them is guessing.
Watch for confident-and-wrong
AIs sound just as sure when they're wrong as when they're right. Just because an AI sounds sure, that doesn't mean it's right.
Ask where it came from
Ask the AI where it got its answer from. If it can't tell you, or if it makes something up, be careful. Don't trust it right away.
These four habits are not a checklist you tick once and forget. They're a way of reading anything an AI hands you.
Ask better, get better
The four habits catch bias after it happens. There's also something we can do BEFORE we press send: write a better question.
AIs fill in the blanks. If we leave the blanks wide open, the AI fills them with whatever pattern it saw most. If we close the blanks, the AI has to try harder, and the answer usually gets better.
One easy rule: say what you want to see, AND say what you don't want it to leave out. The two questions below ask for the same thing. Only one of them steers the AI away from the same old answer.
Name ten famous scientists.
The blanks are wide open. The AI will fall back on the pattern it saw most, which is usually American and European men. You'll get exactly that.
Name ten famous scientists, including at least three women and at least three from outside Europe and America. For each one, give me one line about what they did.
The blanks are closed. The AI has to think harder about who counts. You'll still want to double-check the list, but the answer will be much fairer.