Demographic Parity Assessment

Description

Demographic Parity Assessment checks whether a model gives a favourable outcome, such as a loan approval or an interview invitation, to each demographic group at the same rate. It compares those rates without adjusting for how many people in each group would qualify. Statistical Parity Difference is the gap between group rates (a signed difference between two named groups in AIF360, the highest rate minus the lowest across all groups in Fairlearn), and the Disparate Impact ratio divides the lowest rate by the highest. A ratio below 0.8 is a common warning sign, following the four-fifths rule used by US employment regulators. A gap shows that outcomes differ between groups but not why, and equal rates do not show that individuals were treated fairly.

Example Use Cases

Fairness

A bank's model risk team compares loan approval rates for applicants of different ethnic groups in the validation sample before a credit model goes live. It reports the ratio of the lowest to the highest approval rate and checks it against the 0.8 rule of thumb from US employment practice, treating a lower ratio as a prompt to investigate rather than a verdict.

An employer's HR analytics team tracks each month the share of male and female applicants that a CV-screening tool passes to interview. A widening gap in pass rates triggers a review of the tool before the next hiring round, even where the two applicant pools differ.

A local council checks whether its automated screening tool refers benefit claims for fraud investigation at similar rates across ethnic groups. Because a referral is a burden rather than a benefit, a higher referral rate for one group is a red flag that officers then examine case by case.

Limitations

Parity can be met by unfair selection. A model can match approval rates by choosing less suitable applicants from one group, and the metric looks only at outcome rates, so it cannot tell. Dwork et al. give this as a standard example of why parity alone is not enough.
May penalise a model for real differences between groups in how many members qualify (the base rate), such as differing clinical need by age, and so push towards artificial equality where none should exist.
Can conflict with individual fairness principles, where similarly qualified individuals might receive different treatment to achieve group parity.
A pass on one attribute can hide large gaps inside subgroups, such as older women within a comparison by gender. The common 0.8 ratio threshold is a rule of thumb from US employment practice, so passing it is a screen rather than evidence of compliance in other domains.
A rate ratio depends on which group is the reference and can be unstable when its positive rate is very small. It is undefined when the denominator rate is zero.

Resources

Research Papers

Fairness Through Awareness

Cynthia Dwork et al.•Apr 20, 2011

We study fairness in classification, where individuals are classified, e.g., admitted to a university, and the goal is to prevent discrimination against individuals based on their membership in some group, while maintaining utility for the classifier (the university). The main conceptual contribution of this paper is a framework for fair classification comprising (1) a (hypothetical) task-specific metric for determining the degree to which individuals are similar with respect to the classification task at hand; (2) an algorithm for maximizing utility subject to the fairness constraint, that similar individuals are treated similarly. We also present an adaptation of our approach to achieve the complementary goal of "fair affirmative action," which guarantees statistical parity (i.e., the demographics of the set of individuals receiving any classification are the same as the demographics of the underlying population), while treating similar individuals as similarly as possible. Finally, we discuss the relationship of fairness to privacy: when fairness implies privacy, and how tools developed in the context of differential privacy may be applied to fairness.

Certifying and Removing Disparate Impact

Michael Feldman et al.•Aug 10, 2015

What does it mean for an algorithm to be biased? In U.S. law, unintentional bias is encoded via disparate impact, which occurs when a selection process has widely different outcomes for different groups, even as it appears to be neutral. This legal determination hinges on a definition of a protected class (ethnicity, gender) and an explicit description of the process. When computers are involved, determining disparate impact (and hence bias) is harder. It might not be possible to disclose the process. In addition, even if the process is open, it might be hard to elucidate in a legal setting how the algorithm makes its decisions. Instead of requiring access to the process, we propose making inferences based on the data it uses. We present four contributions. First, we link disparate impact to a measure of classification accuracy that while known, has received relatively little attention. Second, we propose a test for disparate impact based on how well the protected class can be predicted from the other attributes. Third, we describe methods by which data might be made unbiased. Finally, we present empirical evidence supporting the effectiveness of our test for disparate impact and our approach for both masking bias and preserving relevant information in the data. Interestingly, our approach resembles some actual selection practices that have recently received legal scrutiny.

Software Packages

AIF360

Aug 22, 2018

A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.

Tutorials

AIF360 tutorial: Detecting and mitigating age bias on credit decisions (notebook)

A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models. - Trusted-AI/AIF360

Documentations

Fairlearn - Demographic Parity

Fairlearn Developers•Jan 1, 2018

Related Techniques

Name	Description	Assurance Goals
Predictive Parity and Calibration Within Groups	This test checks whether a model's output means the same for every group: among people who receive the same score or the same decision, the share who turn out to have the outcome (or its average, for an outcome such as a cost or a count of illnesses) should not depend on their group, a requirement the fairness literature calls sufficiency. For a model that outputs a risk score, cases with known outcomes are grouped into score bins and, for each group, the observed outcome rate in each bin is plotted against the score, giving one calibration curve per group (a score is calibrated when, of the cases scored 0.3, about 30 per cent have the outcome); calibration within groups (Kleinberg, Mullainathan and Raghavan) holds when each group's curve matches the scores, and the weaker condition that the curves coincide holds when a given score corresponds to the same outcome rate in every group. For a yes/no decision the same comparison is made in the two cells of the decision: predictive parity (Chouldechova) holds when the positive predictive value, the share of people flagged positive who turn out positive, is equal across groups, and the negative predictive value can be compared in the same way for those not flagged. The two read-outs can disagree, because a score that means the same for every group can still give unequal positive predictive values at a threshold when the groups' scores are spread differently above it, so the test reports the read-out matching the output on which decisions are taken, with intervals and the gaps between groups. Where comparisons of error rates condition on the true outcome and ask whether people who turn out the same are treated the same, this test conditions on the model's output and asks whether people treated the same turn out the same; when the outcome is more common in one group, the two cannot both hold except in special cases.	Fairness
Equal Opportunity Difference	Equal Opportunity Difference checks whether a model is equally likely to give a favourable prediction to the people whose true outcome is favourable, such as loan applicants who would repay, in each demographic group. For each group it calculates the true positive rate: of the people whose true outcome is positive, the share the model also predicts as positive. The score is the gap between two groups' rates, usually the unprivileged group's minus the privileged group's, and 0 means the rates match. Some libraries report the largest gap across all groups instead. The metric follows the equality of opportunity criterion of Hardt, Price and Srebro (2016), and it says nothing about people whose true outcome is negative, so a score of 0 does not show that the model is fair overall.	Fairness
Average Odds Difference	Average Odds Difference summarises how differently a model treats two demographic groups, using two error rates. The true positive rate is the share of people whose true outcome is positive whom the model also predicts as positive. The false positive rate is the share of people whose true outcome is negative whom the model wrongly predicts as positive. The score is the mean of the gap in each rate between the groups, usually the unprivileged group minus the privileged group, so 0 means the two gaps are absent or cancel out. Hardt, Price and Srebro (2016) call the stricter requirement that both gaps are zero equalised odds, and a score near 0 does not guarantee it.	Fairness