Conditional Demographic Disparity

Description

Conditional demographic disparity (CDD) measures whether members of a protected group make up a larger share of the people a system turns down than of the people it accepts, after taking into account a factor that differs between groups and that the assessor treats as a legitimate explanation of outcomes; Wachter, Mittelstadt and Russell (2021) propose it as baseline statistical evidence for assessing possible discrimination under EU non-discrimination law. Demographic disparity is the group's share of the unfavourable outcomes minus its share of the favourable ones: in the Berkeley graduate admissions data the paper revisits, women were 46 per cent of rejected applicants but 32 per cent of admitted ones, a disparity of 0.14. CDD computes the same difference separately within each stratum of the conditioning attribute (there, the department applied to) and averages the strata, weighting each by its number of people, so a gap that arises only because groups apply in different proportions to strata with different acceptance rates (an instance of Simpson's paradox) drops out, while a gap within strata remains. Conditioning on department turns the Berkeley figure to about -0.02, a slight disparity in women's favour: within departments, women were on average 40 per cent of rejected and 42 per cent of admitted applicants. The output is the disparity in each stratum and their weighted average, computed on the outcomes recorded in a dataset before training or on a model's decisions after it; the authors present it as a descriptive magnitude to support a contextual judgement, not a pass-or-fail test, and the choice of conditioning attribute is a judgement the measure does not make.

Example Use Cases

Fairness

An employer's HR analytics team audits a CV-screening model used across job families. Women make up a larger share of screened-out applicants than of shortlisted ones overall, but apply in greater numbers to the families with the fewest vacancies per applicant, so the team reports CDD conditioned on job family, with each family's own disparity, letting the recruitment board see whether a gap remains within families before it decides whether to keep using the model.

A consumer lender's compliance team computes CDD on its credit model's decisions for applicants of different ethnic origins, recorded for equality monitoring, conditioned first on the loan product applied for and then on income band. It reports both results with the per-stratum figures and a written reason for treating each factor as a legitimate explanation, as the starting point for its assessment of possible indirect discrimination.

A local authority planning to train a model on past housing allocation decisions measures CDD in those decisions, conditioned on assessed band of housing need, before any training. A positive disparity for one ethnic group within need bands, larger than chance would explain, indicates that the labels carry a pattern a model could learn, and the project board uses the finding to decide whether those decisions can serve as training labels at all.

Limitations

The result depends on which attribute is conditioned on, and the measure cannot tell whether that attribute is a legitimate explanation. If the attribute is itself shaped by discrimination, for instance if one group was deterred from applying to some departments, conditioning on it removes the very disparity a claimant would point to. Wachter et al. leave the choice of factors to courts and context, so each choice needs a recorded justification that a reviewer can contest.
The weighted average can hide large disparities of opposite sign in different strata. In the Berkeley data the per-department disparities run from about -0.09 to +0.05 around a summary of -0.02, and Wachter et al. note that the largest swings come from the department where few women applied, which chance alone could produce. A claim should rest on the per-stratum figures, with their counts, as well as on the summary.
Searching many conditioning attributes or their combinations will find disparities by chance, as the authors warn, and choosing among them after seeing the results can equally make a real one disappear. The authors also offer a descriptive magnitude rather than a statistical test, and set no threshold for how much disparity is too much, so the conditioning attributes need fixing before the data are examined, and a claim that a gap is real needs an interval or test added.
CDD compares shares of outcomes, not rates, so the same gap in acceptance rates gives a smaller value where the group is a small part of a stratum: accepted at 20 per cent against 50 per cent for others, a group scores about 0.33 where it is half the stratum but 0.02 where it is 2 per cent. A fixed threshold therefore tolerates larger rate gaps for smaller groups. Scores and graded outcomes must also first be cut into favourable and unfavourable, and the result changes with the cut.
In the open-source SageMaker Clarify code, a stratum where everyone gets the same outcome is counted rather than skipped: an all-rejected stratum adds a disparity equal to the group's share of it, though no one there was treated differently, so such strata need removing first. AWS has closed Clarify to new customers. The calculation is easy to reimplement, but published applications to deployed AI systems are scarce; the paper's worked example is the 1975 Berkeley admissions study.

Resources

Research Papers

Why fairness cannot be automated: Bridging the gap between EU non-discrimination law and AI

Sandra Wachter, Brent Mittelstadt, and Chris Russell•Jul 1, 2021

Software Packages

smclarify (aws/amazon-sagemaker-clarify)

May 23, 2026

Fairness Aware Machine Learning. Bias detection and mitigation for datasets and models.

Tutorials

SMclarify bias metrics example usage (example notebook)

Fairness Aware Machine Learning. Bias detection and mitigation for datasets and models. - aws/amazon-sagemaker-clarify

Documentations

Conditional Demographic Disparity (CDD) (Amazon SageMaker AI Developer Guide)

Amazon SageMaker Clarify conditional demographic disparity data bias metric.

Related Techniques

Name	Description	Assurance Goals
Demographic Parity Assessment	Demographic Parity Assessment checks whether a model gives a favourable outcome, such as a loan approval or an interview invitation, to each demographic group at the same rate. It compares those rates without adjusting for how many people in each group would qualify. Statistical Parity Difference is the gap between group rates (a signed difference between two named groups in AIF360, the highest rate minus the lowest across all groups in Fairlearn), and the Disparate Impact ratio divides the lowest rate by the highest. A ratio below 0.8 is a common warning sign, following the four-fifths rule used by US employment regulators. A gap shows that outcomes differ between groups but not why, and equal rates do not show that individuals were treated fairly.	Fairness
Predictive Parity and Calibration Within Groups	This test checks whether a model's output means the same for every group: among people who receive the same score or the same decision, the share who turn out to have the outcome (or its average, for an outcome such as a cost or a count of illnesses) should not depend on their group, a requirement the fairness literature calls sufficiency. For a model that outputs a risk score, cases with known outcomes are grouped into score bins and, for each group, the observed outcome rate in each bin is plotted against the score, giving one calibration curve per group (a score is calibrated when, of the cases scored 0.3, about 30 per cent have the outcome); calibration within groups (Kleinberg, Mullainathan and Raghavan) holds when each group's curve matches the scores, and the weaker condition that the curves coincide holds when a given score corresponds to the same outcome rate in every group. For a yes/no decision the same comparison is made in the two cells of the decision: predictive parity (Chouldechova) holds when the positive predictive value, the share of people flagged positive who turn out positive, is equal across groups, and the negative predictive value can be compared in the same way for those not flagged. The two read-outs can disagree, because a score that means the same for every group can still give unequal positive predictive values at a threshold when the groups' scores are spread differently above it, so the test reports the read-out matching the output on which decisions are taken, with intervals and the gaps between groups. Where comparisons of error rates condition on the true outcome and ask whether people who turn out the same are treated the same, this test conditions on the model's output and asks whether people treated the same turn out the same; when the outcome is more common in one group, the two cannot both hold except in special cases.	Fairness
Equal Opportunity Difference	Equal Opportunity Difference checks whether a model is equally likely to give a favourable prediction to the people whose true outcome is favourable, such as loan applicants who would repay, in each demographic group. For each group it calculates the true positive rate: of the people whose true outcome is positive, the share the model also predicts as positive. The score is the gap between two groups' rates, usually the unprivileged group's minus the privileged group's, and 0 means the rates match. Some libraries report the largest gap across all groups instead. The metric follows the equality of opportunity criterion of Hardt, Price and Srebro (2016), and it says nothing about people whose true outcome is negative, so a score of 0 does not show that the model is fair overall.	Fairness