Counterfactual Fairness Assessment

Description

Counterfactual Fairness Assessment tests whether a model would have made the same predictions for a person had their protected attribute, such as race or sex, been different, with everything that attribute influences changed accordingly (Kusner et al., 2017). The assessor first draws a causal graph, a diagram of which variables cause which, covering the protected attribute, the model's inputs and unobserved background factors, and fits it to data. For each person, the fitted graph is used to infer their background factors, switch the attribute to another value and recompute every input it influences, which gives a counterfactual version of that person. The model is run on both versions, and the gap between the two predictions, person by person or as a distance between the two sets of predictions, is the evidence for or against counterfactual fairness under the assumed graph. Every causal effect of the attribute counts as unfair, including effects passing through inputs such as test scores, which separates this criterion from versions that allow some pathways.

Example Use Cases

Fairness

A law school's admissions office uses a model to predict first-year grades from entrance-test scores and undergraduate grades. Its analysts build a causal model in which race and sex influence both inputs alongside an unobserved knowledge factor, fit it to past cohorts, and compare each applicant's predicted grade under actual and counterfactual race. A systematic shift shows that leaving race out of the inputs did not make the model counterfactually fair.

A credit risk team, working with economists, specifies a causal graph in which an applicant's ethnicity affects income and neighbourhood but not repayment behaviour directly, fits it to its applicant data, and checks whether the approval model's scores change for the counterfactual applicants. The result goes to the model risk committee together with the graph, because the conclusion holds only under that graph.

A public employment service reviewing a model that predicts job-seekers' risk of long-term unemployment from education and work history specifies, with labour-market researchers, how sex and migration background influence those inputs. It tests whether individual risk scores would change under counterfactual sex or migration background before the scores are used to allocate support.

Limitations

The causal model can have testable implications, but its counterfactual assumptions are generally not identifiable from observed or experimental data. Plausible alternative models can therefore produce different fairness conclusions.
A common cause left out of the graph, such as school quality affecting both test scores and later grades, biases the counterfactuals without any warning sign. Kilbertus et al. (2019) show that counterfactual fairness is sensitive to such unmeasured confounding and give tools to bound the effect for additive noise models, which adds a further analysis to do.
Every effect of the attribute counts as unfair, so a model that uses an input the attribute influences will usually fail the test, unless those effects cancel out. Kusner et al. note that in practice few observed variables are unaffected by the attribute. Where some pathways are acceptable, for example where qualifications differ for reasons a decision may properly reflect, this criterion is stricter than the decision calls for.
Computing individual counterfactuals requires a fitted probabilistic model of the unobserved background factors; the original study fitted it in Stan, and DoWhy's counterfactual function needs mechanisms with additive noise and supports only continuous variables. With categorical inputs, many variables or unstructured data such as text and images, the assessment becomes hard to run or yields a spread of counterfactuals rather than one.
The counterfactual people are simulated by the fitted causal model, not observed. If its equations fit the data poorly or miss interactions, the counterfactual inputs are implausible and the measured gap reflects the causal model's errors as much as the predictor's; DoWhy's documentation notes counterfactual estimates are approximate for this reason.

Resources

Research Papers

Counterfactual Fairness

Matt J. Kusner et al.•Dec 4, 2017

Machine learning can impact people with legal or ethical consequences when it is used to automate decisions in areas such as insurance, lending, hiring, and predictive policing. In many of these scenarios, previous decisions have been made that are unfairly biased against certain subpopulations, for example those of a particular race, gender, or sexual orientation. Since this past data may be biased, machine learning predictors must account for this to avoid perpetuating or creating discriminatory practices. In this paper, we develop a framework for modeling fairness using tools from causal inference. Our definition of counterfactual fairness captures the intuition that a decision is fair towards an individual if it the same in (a) the actual world and (b) a counterfactual world where the individual belonged to a different demographic group. We demonstrate our framework on a real-world problem of fair prediction of success in law school.

Software Packages

DoWhy: a Python library for causal inference (py-why/dowhy)

Oct 7, 2026

DoWhy is a Python library for causal inference that supports explicit modeling and testing of causal assumptions. DoWhy is based on a unified language for causal inference, combining causal graphical models and potential outcomes frameworks.

Code for 'Counterfactual Fairness' (NIPS 2017) (mkusner/counterfactual-fairness)

Aug 11, 2026

Code for "Counterfactual Fairness" (NIPS2017)

Tutorials

DoWhy example notebook: Counterfactual Fairness

Documentations

DoWhy user guide: Computing Counterfactuals

Related Techniques

Name	Description	Assurance Goals
Path-Specific Counterfactual Fairness Assessment	Path-specific counterfactual fairness assessment tests whether a protected attribute, such as sex or race, influences a decision along routes that the assessor judges unfair. The assessor first draws a causal graph, a diagram of which variables influence which, and marks each route from the protected attribute to the decision as acceptable or unacceptable. In a classic admissions example, women applied more often to departments with low admission rates, so the effect of sex through department choice was treated as acceptable, while any direct effect of sex was not. The technique then estimates how the decision would change if the protected attribute were switched, counting only the effect that travels along the unacceptable routes. A large estimate points to unfair influence along those routes, and the estimate is only as sound as the graph and the choice of routes.	Fairness
Causal Discrimination Testing	Causal discrimination testing checks whether a system gives different outputs to otherwise identical inputs because of a protected attribute, by changing only that attribute, or a name, term or other proxy that signals it, and measuring how often or how much the output changes. Galhotra, Brun and Meliou (2017), who called the approach fairness testing, generate inputs from a description of the valid input fields, make copies of each that differ only in the protected attribute (for example, recorded sex), run them all through the system as a black box, and report the causal discrimination score: the share of inputs whose decision changes, sampled until it is known within a chosen margin of error and confidence level. For text, the same test substitutes identity terms or names in otherwise unchanged sentences, such as one religion or ethnicity for another, and reports the average change in the model's score across the substitutions, which Garg et al. (2019) call the counterfactual token fairness gap. Because every other part of the input is held fixed, the test needs no model of how the attribute influences other variables: it measures whether the system itself responds to the attribute, or to the chosen proxy. The output is a score per attribute and, if the tester logs them, the input pairs whose outputs differed.	Fairness
Demographic Parity Assessment	Demographic Parity Assessment checks whether a model gives a favourable outcome, such as a loan approval or an interview invitation, to each demographic group at the same rate. It compares those rates without adjusting for how many people in each group would qualify. Statistical Parity Difference is the gap between group rates (a signed difference between two named groups in AIF360, the highest rate minus the lowest across all groups in Fairlearn), and the Disparate Impact ratio divides the lowest rate by the highest. A ratio below 0.8 is a common warning sign, following the four-fifths rule used by US employment regulators. A gap shows that outcomes differ between groups but not why, and equal rates do not show that individuals were treated fairly.	Fairness