Equal Opportunity Difference

Description

Equal Opportunity Difference checks whether a model is equally likely to give a favourable prediction to the people whose true outcome is favourable, such as loan applicants who would repay, in each demographic group. For each group it calculates the true positive rate: of the people whose true outcome is positive, the share the model also predicts as positive. The score is the gap between two groups' rates, usually the unprivileged group's minus the privileged group's, and 0 means the rates match. Some libraries report the largest gap across all groups instead. The metric follows the equality of opportunity criterion of Hardt, Price and Srebro (2016), and it says nothing about people whose true outcome is negative, so a score of 0 does not show that the model is fair overall.

Example Use Cases

Fairness

A bank's model risk team evaluates a loan approval model on past loans with known repayment outcomes. It measures whether applicants who went on to repay were approved at the same rate in each ethnic group, for example 90% for one group against a markedly lower rate for another, before the model is released.

A hospital evaluating a model that flags chest X-rays for suspected pneumonia compares its sensitivity, the share of confirmed cases it flags, between male and female patients and between age bands on a labelled validation set. A gap means affected patients in one group are more likely to be missed, which the clinical safety board weighs before deployment.

A university department checks a model that flags first-year students at risk of dropping out so they can be offered support. Using last year's records, it measures what share of students who actually dropped out were flagged, separately for students who are the first in their family to attend university and for other students.

Limitations

Only considers true positive rates and ignores false positive rate disparities, potentially allowing discrimination in the form of unequal false alarm rates between groups.
The true outcome labels must be reliable for every group. Where they record past decisions, such as repayment known only for applicants previously approved, or are measured differently between groups, equal true positive rates can reproduce the bias already in the labels.
True positive rates use only the people whose true outcome is positive, so a rare outcome leaves few cases in each group. Gaps can then be noise, and splitting by several attributes at once, such as age within gender, makes this worse.
Closing a gap by lowering one group's decision threshold raises that group's true positive rate but also its false positive rate. Fixing this metric can therefore open a gap in another, and the metric gives no warning when it does.
A score of 0 does not show that the model is fair. The measure uses only predictions, true outcomes and group membership, and Hardt et al. show that situations with different fairness readings can give identical numbers, so a result needs scrutiny of how the outcome was produced.

Resources

Research Papers

Equality of Opportunity in Supervised Learning

Moritz Hardt, Eric Price, and Nathan Srebro•Oct 7, 2016

We propose a criterion for discrimination against a specified sensitive attribute in supervised learning, where the goal is to predict some target based on available features. Assuming data about the predictor, target, and membership in the protected group are available, we show how to optimally adjust any learned predictor so as to remove discrimination according to our definition. Our framework also improves incentives by shifting the cost of poor classification from disadvantaged groups to the decision maker, who can respond by improving the classification accuracy. In line with other studies, our notion is oblivious: it depends only on the joint statistics of the predictor, the target and the protected attribute, but not on interpretation of individualfeatures. We study the inherent limits of defining and identifying biases based on such oblivious measures, outlining what can and cannot be inferred from different oblivious tests. We illustrate our notion using a case study of FICO credit scores.

AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias

R. K. E. Bellamy et al.•Jul 1, 2019

Fairness is an increasingly important concern as machine learning models are used to support decision making in high-stakes applications such as mortgage lending, hiring, and prison sentencing. This article introduces a new open-source Python toolkit for algorithmic fairness, AI Fairness 360 (AIF360), released under an Apache v2.0 license (https://github.com/ibm/aif360). The main objectives of this toolkit are to help facilitate the transition of fairness research algorithms for use in an industrial setting and to provide a common framework for fairness researchers to share and evaluate algorithms. The package includes a comprehensive set of fairness metrics for datasets and models, explanations for these metrics, and algorithms to mitigate bias in datasets and models. It also includes an interactive Web experience that provides a gentle introduction to the concepts and capabilities for line-of-business users, researchers, and developers to extend the toolkit with their new algorithms and improvements and to use it for performance benchmarking. A built-in testing infrastructure maintains code quality.

Software Packages

AIF360

Aug 22, 2018

A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.

fairlearn

May 15, 2018

A Python package to assess and improve fairness of machine learning models.

Tutorials

AIF360 tutorial: Medical Expenditure (notebook)

Documentations

aif360.sklearn.metrics.equal_opportunity_difference — aif360 0.6.1 ...

Aif360 Developers•Jan 1, 2018

fairlearn.metrics.equal_opportunity_difference — Fairlearn documentation

Related Techniques

Name	Description	Assurance Goals
Average Odds Difference	Average Odds Difference summarises how differently a model treats two demographic groups, using two error rates. The true positive rate is the share of people whose true outcome is positive whom the model also predicts as positive. The false positive rate is the share of people whose true outcome is negative whom the model wrongly predicts as positive. The score is the mean of the gap in each rate between the groups, usually the unprivileged group minus the privileged group, so 0 means the two gaps are absent or cancel out. Hardt, Price and Srebro (2016) call the stricter requirement that both gaps are zero equalised odds, and a score near 0 does not guarantee it.	Fairness
Demographic Parity Assessment	Demographic Parity Assessment checks whether a model gives a favourable outcome, such as a loan approval or an interview invitation, to each demographic group at the same rate. It compares those rates without adjusting for how many people in each group would qualify. Statistical Parity Difference is the gap between group rates (a signed difference between two named groups in AIF360, the highest rate minus the lowest across all groups in Fairlearn), and the Disparate Impact ratio divides the lowest rate by the highest. A ratio below 0.8 is a common warning sign, following the four-fifths rule used by US employment regulators. A gap shows that outcomes differ between groups but not why, and equal rates do not show that individuals were treated fairly.	Fairness
Predictive Parity and Calibration Within Groups	This test checks whether a model's output means the same for every group: among people who receive the same score or the same decision, the share who turn out to have the outcome (or its average, for an outcome such as a cost or a count of illnesses) should not depend on their group, a requirement the fairness literature calls sufficiency. For a model that outputs a risk score, cases with known outcomes are grouped into score bins and, for each group, the observed outcome rate in each bin is plotted against the score, giving one calibration curve per group (a score is calibrated when, of the cases scored 0.3, about 30 per cent have the outcome); calibration within groups (Kleinberg, Mullainathan and Raghavan) holds when each group's curve matches the scores, and the weaker condition that the curves coincide holds when a given score corresponds to the same outcome rate in every group. For a yes/no decision the same comparison is made in the two cells of the decision: predictive parity (Chouldechova) holds when the positive predictive value, the share of people flagged positive who turn out positive, is equal across groups, and the negative predictive value can be compared in the same way for those not flagged. The two read-outs can disagree, because a score that means the same for every group can still give unequal positive predictive values at a threshold when the groups' scores are spread differently above it, so the test reports the read-out matching the output on which decisions are taken, with intervals and the gaps between groups. Where comparisons of error rates condition on the true outcome and ask whether people who turn out the same are treated the same, this test conditions on the model's output and asks whether people treated the same turn out the same; when the outcome is more common in one group, the two cannot both hold except in special cases.	Fairness