Predictive Parity and Calibration Within Groups
Description
This test checks whether a model's output means the same for every group: among people who receive the same score or the same decision, the share who turn out to have the outcome (or its average, for an outcome such as a cost or a count of illnesses) should not depend on their group, a requirement the fairness literature calls sufficiency. For a model that outputs a risk score, cases with known outcomes are grouped into score bins and, for each group, the observed outcome rate in each bin is plotted against the score, giving one calibration curve per group (a score is calibrated when, of the cases scored 0.3, about 30 per cent have the outcome); calibration within groups (Kleinberg, Mullainathan and Raghavan) holds when each group's curve matches the scores, and the weaker condition that the curves coincide holds when a given score corresponds to the same outcome rate in every group. For a yes/no decision the same comparison is made in the two cells of the decision: predictive parity (Chouldechova) holds when the positive predictive value, the share of people flagged positive who turn out positive, is equal across groups, and the negative predictive value can be compared in the same way for those not flagged. The two read-outs can disagree, because a score that means the same for every group can still give unequal positive predictive values at a threshold when the groups' scores are spread differently above it, so the test reports the read-out matching the output on which decisions are taken, with intervals and the gaps between groups. Where comparisons of error rates condition on the true outcome and ask whether people who turn out the same are treated the same, this test conditions on the model's output and asks whether people treated the same turn out the same; when the outcome is more common in one group, the two cannot both hold except in special cases.
Example Use Cases
Fairness
A hospital's clinical informatics team checks a deterioration-risk score before it is shown to ward staff. On a year of admissions with recorded outcomes it plots a calibration curve, with intervals, for each ethnic group and each sex, and approves the score for use with every patient only if, across the scores that trigger escalation, each group's curve stays within a band agreed with clinicians.
A lender's model validation team compares the repayment rate among approved applicants across age bands, using two years of outcomes, to check whether approval means the same for each band. Because a difference among the approved can come from how scores are spread above the cut-off rather than from a different bar for one band, it also compares the calibration curves of the underlying score before raising any finding with the credit risk committee.
A probation service assessing a supplier's reoffending-risk tool is shown evidence that its positive predictive value is equal across ethnic groups. Its analysts reproduce the figures on local data and set them beside the groups' false positive and false negative rates, which differ because reconviction rates differ, so that the oversight board chooses which criterion the tool will be held to with both sets of figures in front of it.
A health insurer auditing a score that selects members for a care-management programme checks twice whether the score means the same for every racial group: against next year's costs, the label the model was trained on, and against a measure of illness such as the number of active chronic conditions. A score calibrated for cost but not for illness, with one group sicker than another at the same score, would lead the insurer to retrain on a different label.
Limitations
- When the outcome is more common in one group, a score or decision that satisfies predictive parity cannot also give the groups both equal false positive and equal false negative rates, and calibration within groups cannot coexist with equal average scores for each group's positives and negatives except with perfect prediction or equal base rates (Chouldechova; Kleinberg et al.). Passing this test therefore means accepting unequal error rates, a choice the test informs but cannot make.
- Passing says little about how accurately the model predicts for each group. A well-fitted model usually comes close to calibration within groups without any fairness intervention when group membership can be inferred from its inputs, and a model that predicts accurately for one group while giving everyone in another group that group's average rate is still perfectly calibrated (Barocas, Hardt and Narayanan). The result needs pairing with performance measured for each group.
- The test uses recorded outcomes, so it inherits any difference between groups in how the outcome is recorded. Rearrest stands in for reoffending, and a care-management score was calibrated by race for health costs while, at the 97th percentile of the score, Black patients had 26 per cent more chronic illnesses than white patients (Obermeyer et al.). A model can pass while scoring one group's need too low, so the outcome tested must be the one the claim is about.
- A gap in positive predictive value is not by itself evidence that the score treats groups differently: the value averages over everyone above the threshold, so it depends on how each group's scores are spread there and moves when the threshold moves. Chouldechova notes that well-calibrated scores can fail predictive parity, and the converse also holds. Where scores are available the calibration curves should be reported, and the threshold stated.
- Per-group calibration curves need labelled cases across the score range in every group; in a small group the high-score bins may hold a handful of people, so their rates swing widely, and the choice of bin widths changes the picture. An outside auditor who sees only decisions, not scores, can test predictive parity but not calibration, so access determines which claim the evidence can support.