Bias in AI
Why AI systems land unfairly on real people, and how to catch it before it catches you
The CV filter that quietly rejected the women
Before any of the theory, let's watch the pattern happen live.
Pick one of the prompts below and paste it into any image AI you can access: ChatGPT (image mode), Gemini, Microsoft Copilot Designer, whichever you can get through. Three images per prompt is plenty for the skew to jump out, and it keeps free-tier quotas happy.
Look at the results. Try to describe the pattern before you reach for a cause.
You've just watched an image AI have a go at picturing a doctor, and most of them came back looking suspiciously similar. That isn't a rendering quirk. It's the same shape of problem that got Amazon to quietly scrap its own AI CV-screening tool back in 2018, after the tool taught itself to mark down applications that mentioned things like 'women's chess club captain'. The engineers hadn't asked it to do that. It picked the habit up on its own, from ten years of mostly-male CVs.
Bias in AI isn't the AI being prejudiced. It's the AI being an unusually faithful mirror of whatever data it was trained on, whichever team happened to build it, and however people end up using it. Three different places it can hide, and all three can land on real people in ways that are hard to see and even harder to appeal.
Over the next forty minutes we'll take those three flavours apart, look at real cases where AI bias hit people who had no idea it was happening, and finish with a habit-set you can use tomorrow whenever an AI hands you an answer that looks a bit too clean.
Please generate three images of a doctor at work in a hospital. Photorealistic, but each one a bit different.
Please generate three images of a CEO at their desk in a modern office. Photorealistic, but each one a bit different.
Please generate three images of a software engineer writing code at their computer. Photorealistic, but each one a bit different.
Look at the results together and ask the class:
- Who's in the pictures? What do they look like: age, gender, ethnicity, dress?
- Who's missing? If you can name a real doctor, CEO or engineer who breaks the AI's pattern, does the AI know they exist?
- If this same AI were plugged into a hiring platform, a facial-recognition system, or a risk-scoring tool used on real people, how would the pattern land?
- Where do you reckon the AI got its idea of what those three jobs look like?
The mirror in the machine
Every modern AI system learns from a huge pile of examples. For a large language model that's roughly the readable internet, plus books, code, and licensed data. For an image AI it's billions of image-caption pairs. For a hiring tool it might be ten years of the company's own CVs. Whatever the pile is, the AI's job during training is to spot the patterns in it and get better at continuing them.
So when we ask it to produce a doctor, it isn't asking itself a moral question about who ought to be a doctor. It's playing back the pattern it was shown most. If the pile leaned heavily one way, the output leans the same way, only faster and at scale.
That gives us the working definition worth carrying through the rest of the course: AI bias is a systematic skew in an AI's outputs that reflects a skew somewhere in its training data, its design, or how it's being used. Not a glitch, not a bug, not the AI 'being prejudiced'. The unfairness is honest. The mirror is behaving exactly as designed. Which is precisely what makes it a problem worth taking seriously.
Three flavours of bias
It really helps to name which flavour of bias you're looking at, because the fix is different for each one. Bias can enter a system in three places, and real cases usually involve at least two of them at once.
Before you tap the cards below, take a guess. If you were sketching the pipeline from 'raw data' to 'AI decision landing on a real person', where along that pipeline could unfairness sneak in?
Tap each card to reveal.
Data bias
The training data was skewed before the AI ever saw it. Amazon's scrapped CV screener is the textbook case: ten years of mostly-male CVs taught the model that 'male-coded' words correlated with 'good hire'. The AI wasn't wrong about the data. The data was the problem.
Design bias
The people building the AI made choices that quietly excluded some users. The classic study here is Joy Buolamwini and Timnit Gebru's Gender Shades work in 2018: three commercial face-classification systems had error rates of under 1% on lighter-skinned men, but over 20% on darker-skinned women. The training data hadn't been checked against a diverse benchmark, and no one on the build team spotted the gap.
Use bias
The AI works as designed, but a human uses it in a way that produces an unfair result. Example: a university admissions office runs the same AI screener on every UCAS form, but only chases up detailed reference checks for applicants from certain schools. The AI is neutral. The workflow around it isn't.
The first two hide inside the AI. The third hides in the humans around it. When a real scandal breaks, expect to find at least two of these tangled together. Naming the dominant one is the first step to actually fixing anything.
It's already happening to real people
A little skew doesn't sound catastrophic. Nine out of ten doctors, 5% worse for one accent. It sounds like a rounding error you could live with.
The reason it matters is scale. When a system that's a little bit skewed gets deployed to make thousands or millions of decisions a day, that little skew stops being noise and starts being who gets shortlisted for a job, whose loan application clears, whose face unlocks the door, and, in the UK in the summer of 2020, whose A-level grades got downgraded by an algorithm that leaned harder on postcode and school history than on the individual pupil in front of it.
That 2020 A-level algorithm is worth sitting with. It affected pupils exactly like the ones in your classroom now. The government scrapped it within days after public outcry, but by then plenty of offers had already been withdrawn. And the important bit for our purposes: the algorithm was, technically, doing what it was told. The unfairness came from the choice of what to tell it. That's bias.
Four habits to catch it
You genuinely don't need a PhD in machine learning to catch most of the AI bias you'll meet in day-to-day study. Four habits, applied properly, will catch a striking amount of it.
Before you tap the cards, have a guess: if you were coaching a friend who was about to use AI for a UCAS personal statement, an EPQ, or a coursework brief, what would you tell them to watch out for?
Tap each card to reveal.
Ask who's missing
For every answer the AI gives, spend a beat picturing what it didn't say. Which people, ideas, countries, viewpoints, sources? A missing group is almost always more diagnostic than what's in front of you.
Compare two AIs
Run the same prompt through two different systems (say ChatGPT and Gemini). If they meaningfully disagree, at least one of them is guessing, and probably both are. Consensus is a weak signal too, because they often share training data, but visible disagreement is a red flag worth chasing.
Watch for confident-and-wrong
AI systems produce the same fluent, self-assured tone whether they're right or making it up. A calm and specific answer is not evidence of accuracy. Treat confidence as decoration, not proof.
Question the source
Ask yourself: what was this model trained on, roughly, and does that pile of data know about what you're asking? A model trained mostly on English-language internet in 2023 won't have a balanced view of, say, Nigerian literary criticism or last week's news.
These four aren't a checklist you tick and forget. They're a reading practice. Every AI output becomes a text to read critically, not a fact to accept. Applied consistently, they close most of the everyday gap.
Bad prompt, good prompt
The four habits catch bias after it appears. There's also a trick you can use before you even hit return: write a better prompt.
AI systems fill in blanks. If your prompt leaves the blanks wide open, the AI fills them with whatever pattern the training data pushed hardest. If your prompt closes the blanks explicitly, the AI has to work harder, and the answer usually improves.
The two prompts below ask for effectively the same thing. Only one of them steers the AI away from its default skew.
Give me a list of ten of the most important inventors in history, with a short line about each.
Wide-open blanks. The AI defaults to whichever pattern its training data leaned hardest on, which is well-documented as heavily American and European men. You'll get exactly that, and it will sound authoritative.
Give me a list of ten of the most important inventors in history. Include at least three women, cover at least four different continents, and include at least one invention from before the 18th century. For each, give me a one-sentence contribution and a rough date. Prioritise inventors who are historically important rather than currently famous.
The blanks are closed. Geography, gender, era and criterion are all specified, so the AI has to search a wider set of candidates. You still need to fact-check every entry, but the starting list is dramatically more balanced and useful.
So what would you do?
Let's put everything together on a scenario that could easily happen to a Y12 in the next month.
You're working on an EPQ (or a UCAS personal statement, or an A-level History NEA, take your pick). You ask ChatGPT for a list of the ten most influential thinkers in your topic. Back comes a confident, well-written list. All ten are men. Nine are from Western Europe or the US. Zero are from before 1900.
You know enough about your topic to be uneasy. You have the four habits AND the good-prompt trick. Marker's deadline is in two weeks. You need a plan.