21.2 Bias in AI

Bias in AI happens when a system produces systematically unfair outcomes for certain groups of people. It usually comes not from the algorithm 'deciding to be unfair', but from biased training data, skewed labels, or design choices that reflect existing inequalities in society.

Bias often enters through the data pipeline — collection, labeling, and sampling — long before the model is trained.

For example, a speech-recognition app trained mostly on Addis Ababa accents may perform poorly for speakers from Tigray, Oromia, or the Somali region. The model is not 'broken' technically — it simply never saw enough diverse data, so it serves some Ethiopians worse than others.

Scenario

You audit a loan model and find it approves urban applicants far more often than rural ones with similar incomes. What is the best first action?

Check your understanding

1/4 · 40 XP

What is the most common source of bias in AI systems?