Bias in AI
Why AI systems land unfairly on real people, and how to catch it before it catches you
The CV filter that quietly rejected the women
Before any of the theory, let's watch the pattern happen live.
Pick one of the prompts below and paste it into any image AI you can get to on the classroom screen: ChatGPT (image mode), Gemini, Microsoft Copilot Designer, whichever the school firewall lets through. Three images per prompt is plenty for the skew to jump out, and it keeps free-tier quotas happy.
Project the results and then look at them together. Try not to tell the class what to notice. Let them tell you.
You've just watched an image AI have a go at picturing a doctor, and most of them came back looking suspiciously similar. That isn't a rendering quirk. It's the same shape of problem that got Amazon to quietly scrap its own AI CV-screening tool back in 2018, after the tool taught itself to mark down applications that mentioned things like 'women's chess club captain'. The engineers hadn't asked it to do that. It picked the habit up on its own, from ten years of mostly-male CVs.
Bias in AI isn't the AI being prejudiced. It's the AI being an unusually faithful mirror of whatever data it was trained on, whichever team happened to build it, and however people end up using it. Three different places it can hide, and all three can land on real people in ways that are hard to see and even harder to appeal.
Over the next forty minutes we'll take those three flavours apart, look at real cases where AI bias hit people who had no idea it was happening, and finish with a habit-set you can use tomorrow whenever an AI hands you an answer that looks a bit too clean.
Please generate three images of a doctor at work in a hospital. Photorealistic, but each one a bit different.
Please generate three images of a CEO at their desk in a modern office. Photorealistic, but each one a bit different.
Please generate three images of a software engineer writing code at their computer. Photorealistic, but each one a bit different.
Look at the results together and ask the class:
- Who's in the pictures? What do they look like: age, gender, ethnicity, dress?
- Who's missing? If you can name a real doctor, CEO or engineer who breaks the AI's pattern, does the AI know they exist?
- If this same AI were plugged into a hiring platform, a facial-recognition system, or a risk-scoring tool used on real people, how would the pattern land?
- Where do you reckon the AI got its idea of what those three jobs look like?
Before you generate anything, get the class to write down on paper: 'If I asked an AI to draw a doctor, I think it would show me a person who looks like...'. Same for CEO and engineer. Fold the papers.
Generate the images. Now unfold. How many pupils correctly predicted the skew? That prediction itself is worth talking about: the class already knew what the AI would do, which means the pattern isn't hidden. It's just quietly baked in.
The mirror in the machine
Every modern AI system learns from a huge pile of examples. For a large language model that's roughly the readable internet, plus books, code, and licensed data. For an image AI it's billions of image-caption pairs. For a hiring tool it might be ten years of the company's own CVs. Whatever the pile is, the AI's job during training is to spot the patterns in it and get better at continuing them.
So when we ask it to produce a doctor, it isn't asking itself a moral question about who ought to be a doctor. It's playing back the pattern it was shown most. If the pile leaned heavily one way, the output leans the same way, only faster and at scale.
That gives us the working definition worth carrying through the rest of the course: AI bias is a systematic skew in an AI's outputs that reflects a skew somewhere in its training data, its design, or how it's being used. Not a glitch, not a bug, not the AI 'being prejudiced'. The unfairness is honest. The mirror is behaving exactly as designed. Which is precisely what makes it a problem worth taking seriously.
In pairs, pupils sketch on paper the ideal training set for an AI that generates realistic UK doctors. How many images? Age spread? Gender spread? Ethnicity? Junior, consultant, GP, paediatric, surgical? Rural and urban? What percentage of each?
Share three pairs' lists at the front. The point that lands: even careful, well-meaning humans have to make hundreds of quiet choices, and every one of those choices becomes a pattern the AI will faithfully copy. Now imagine the same job done in a rush by a team of engineers under a delivery deadline. That's how most real training sets get made.
Three flavours of bias
It really helps to name which flavour of bias you're looking at, because the fix is different for each one. Bias can enter a system in three places, and real cases usually involve at least two of them at once.
Before you tap the cards below, take a guess. If you were sketching the pipeline from 'raw data' to 'AI decision landing on a real person', where along that pipeline could unfairness sneak in?
Tap each card to reveal.
Data bias
The training data was skewed before the AI ever saw it. Amazon's scrapped CV screener is the textbook case: ten years of mostly-male CVs taught the model that 'male-coded' words correlated with 'good hire'. The AI wasn't wrong about the data. The data was the problem.
Design bias
The people building the AI made choices that quietly excluded some users. The classic study here is Joy Buolamwini and Timnit Gebru's Gender Shades work in 2018: three commercial face-classification systems had error rates of under 1% on lighter-skinned men, but over 20% on darker-skinned women. The training data hadn't been checked against a diverse benchmark, and no one on the build team spotted the gap.
Use bias
The AI works as designed, but a human uses it in a way that produces an unfair result. Example: a university admissions office runs the same AI screener on every UCAS form, but only chases up detailed reference checks for applicants from certain schools. The AI is neutral. The workflow around it isn't.
The first two hide inside the AI. The third hides in the humans around it. When a real scandal breaks, expect to find at least two of these tangled together. Naming the dominant one is the first step to actually fixing anything.
In pairs, pupils find one real example of AI or algorithmic bias from the last few years. Any credible source is fine: BBC, Guardian, Ada Lovelace Institute, a research paper's abstract. They should name the case in one sentence and identify which flavour of bias is dominant.
Collect four or five at the front. Deliberately pick cases where the class disagrees on the flavour, and let them argue it out. Almost every real case involves more than one flavour, and the argument itself is the learning.
It's already happening to real people
A little skew doesn't sound catastrophic. Nine out of ten doctors, 5% worse for one accent. It sounds like a rounding error you could live with.
The reason it matters is scale. When a system that's a little bit skewed gets deployed to make thousands or millions of decisions a day, that little skew stops being noise and starts being who gets shortlisted for a job, whose loan application clears, whose face unlocks the door, and, in the UK in the summer of 2020, whose A-level grades got downgraded by an algorithm that leaned harder on postcode and school history than on the individual pupil in front of it.
That 2020 A-level algorithm is worth sitting with. It affected pupils exactly like the ones in your classroom now. The government scrapped it within days after public outcry, but by then plenty of offers had already been withdrawn. And the important bit for our purposes: the algorithm was, technically, doing what it was told. The unfairness came from the choice of what to tell it. That's bias.
Motion: An AI decision system that's fast, cheap, and slightly biased is better than a slow, expensive, human-only process that's also slightly biased.
Split the class in half. Four minutes to prepare, six minutes to hear both sides. Push pupils to name specific domains where they'd argue differently: shortlisting a job, grading an exam, approving a mortgage, diagnosing a lung scan, sentencing a defendant. Ask them: is the answer the same for all five? Take a vote after the debate, not before, and see how many minds moved.
Four habits to catch it
You genuinely don't need a PhD in machine learning to catch most of the AI bias you'll meet in day-to-day study. Four habits, applied properly, will catch a striking amount of it.
Before you tap the cards, have a guess: if you were coaching a friend who was about to use AI for a UCAS personal statement, an EPQ, or a coursework brief, what would you tell them to watch out for?
Tap each card to reveal.
Ask who's missing
For every answer the AI gives, spend a beat picturing what it didn't say. Which people, ideas, countries, viewpoints, sources? A missing group is almost always more diagnostic than what's in front of you.
Compare two AIs
Run the same prompt through two different systems (say ChatGPT and Gemini). If they meaningfully disagree, at least one of them is guessing, and probably both are. Consensus is a weak signal too, because they often share training data, but visible disagreement is a red flag worth chasing.
Watch for confident-and-wrong
AI systems produce the same fluent, self-assured tone whether they're right or making it up. A calm and specific answer is not evidence of accuracy. Treat confidence as decoration, not proof.
Question the source
Ask yourself: what was this model trained on, roughly, and does that pile of data know about what you're asking? A model trained mostly on English-language internet in 2023 won't have a balanced view of, say, Nigerian literary criticism or last week's news.
These four aren't a checklist you tick and forget. They're a reading practice. Every AI output becomes a text to read critically, not a fact to accept. Applied consistently, they close most of the everyday gap.
Split the class in half. Give both halves the same prompt: 'Name ten of the most influential scientists of the 20th century, with a one-line contribution for each.' One half runs it in ChatGPT, the other in Gemini or Copilot.
Bring both lists up at the front, side by side. Highlight every difference. Ask the class: whose list is right? Answer: neither is a source of truth, and the disagreement itself is habit two in action. Push further: who's missing from both lists? What kinds of scientist would you have expected but don't see? That's habit one.
Bad prompt, good prompt
The four habits catch bias after it appears. There's also a trick you can use before you even hit return: write a better prompt.
AI systems fill in blanks. If your prompt leaves the blanks wide open, the AI fills them with whatever pattern the training data pushed hardest. If your prompt closes the blanks explicitly, the AI has to work harder, and the answer usually improves.
The two prompts below ask for effectively the same thing. Only one of them steers the AI away from its default skew.
Give me a list of ten of the most important inventors in history, with a short line about each.
Wide-open blanks. The AI defaults to whichever pattern its training data leaned hardest on, which is well-documented as heavily American and European men. You'll get exactly that, and it will sound authoritative.
Give me a list of ten of the most important inventors in history. Include at least three women, cover at least four different continents, and include at least one invention from before the 18th century. For each, give me a one-sentence contribution and a rough date. Prioritise inventors who are historically important rather than currently famous.
The blanks are closed. Geography, gender, era and criterion are all specified, so the AI has to search a wider set of candidates. You still need to fact-check every entry, but the starting list is dramatically more balanced and useful.
In pairs, hand pupils one of the three prompts below. Their job: rewrite it so it steers the AI away from its default skew, with at least one 'include' clause and one 'across' or 'from at least' clause.
- Name ten of the most famous scientists.
- Write a story about a nurse having a busy shift on a hospital ward.
- Suggest five careers a Y11 pupil interested in engineering might consider.
Share the two strongest rewrites. Test one live at the front and let the class judge whether the rewrite actually shifted the AI's output. Sometimes it will; sometimes the AI will resist. Both outcomes are worth talking about.
So what would you do?
Let's put everything together on a scenario that could easily happen to a Y12 in the next month.
You're working on an EPQ (or a UCAS personal statement, or an A-level History NEA, take your pick). You ask ChatGPT for a list of the ten most influential thinkers in your topic. Back comes a confident, well-written list. All ten are men. Nine are from Western Europe or the US. Zero are from before 1900.
You know enough about your topic to be uneasy. You have the four habits AND the good-prompt trick. Marker's deadline is in two weeks. You need a plan.
Read the four options below out loud. Hands up on which pupils would pick. Then ask three pupils to justify: one for the strongest option, one for the tempting-but-weak option, and one for whichever the class actually voted for.
Push them to name which habit or prompt-rule they're using at each step. The goal isn't consensus, it's making the reasoning visible. A pupil who picks the wrong option for a good reason is doing better learning than one who picks the right option for a shrug.