Threshold Optimiser
Description
Threshold Optimiser is a post-processing method, introduced by Hardt, Price and Srebro (2016), that adjusts a trained classifier's decisions to meet a group fairness criterion without retraining it, most often equalised odds (equal true positive and false positive rates in every group) or equal opportunity (equal true positive rates only). It uses a labelled validation set that records each person's group, and traces each group's ROC curve, which shows the true and false positive rates the model's score gives at every possible cut-off. It then chooses a separate cut-off for each group that keeps accuracy, or another chosen objective, as high as the criterion allows. Equalised odds usually cannot be met with one cut-off per group, so each group's decision is randomised between two cut-offs with fixed probabilities. The output is a rule that needs each person's group at prediction time, and if the model gives only yes/no predictions, a small linear optimisation sets the probability of flipping each prediction instead.
Example Use Cases
Fairness
A lender's model validation team applies the method to an existing credit-scoring model, choosing a separate approval cut-off for each racial group so that applicants who go on to repay are approved at the same rate in every group (equal opportunity). It compares the expected profit of that rule with a single cut-off and with an equalised-odds rule before recommending which one the credit committee should adopt.
A hospital system's data science team adjusts the alert threshold of an unplanned-readmission model separately for each race and ethnicity group, so that patients who are later readmitted are flagged at similar rates across groups. The change is accepted only because overall accuracy and daily alert volume stay within limits agreed with clinicians before the adjustment was made.
A public agency licenses a pre-trial risk tool that returns only high or low labels, with no access to scores or retraining. Using a held-out set of past cases with known outcomes, it derives the label-flipping rule that gives equal true and false positive rates across racial groups, and reports to its oversight board how many decisions the rule would change and how much accuracy it costs.
An HR analytics team piloting a CV-scoring model runs the method under equal opportunity and under equalised odds on its validation data, to show the selection panel what each criterion costs in accuracy and how many shortlisting decisions it would change. The comparison is the evidence for which fairness criterion the panel signs off before deployment.
Limitations
- The cut-off depends on the person's group, so the protected attribute must be available at the moment of decision (Fairlearn's predict requires it). Where the attribute is not collected, or where applying different cut-offs by group could count as direct discrimination or disparate treatment, which Fairlearn's own guide flags for EU and US law, the rule may be unusable in deployment even though it works in validation.
- Meeting equalised odds exactly usually requires randomisation, so two people with the same score and group can receive different decisions, and Fairlearn notes that some solutions give a non-zero chance of a positive decision whatever the score. Where decisions must be reproducible or explained to the individual, such as credit refusals, this can rule the method out or force a relaxed constraint.
- Under equalised odds every group's error rates are held to what the hardest-to-predict group's ROC curve allows; Hardt et al. note such a classifier must classify everyone as poorly as the hardest group. In their credit case study it kept about 80% of maximum profit against about 93% for equal opportunity, and Fairlearn warns of levelling down: worse performance for the better-served group with no gain for the worse-served one.
- Cut-offs are estimated from a finite labelled sample in which group membership is known, and the original analysis sets sampling error aside. Small groups give unstable cut-offs, and rates equalised on the validation set drift when the population or the model's score distribution changes, so the cut-offs must be re-estimated and the equalised rates monitored after deployment.
- Equalised odds generally conflicts with calibration within groups: Pleiss et al. (2017) show that when groups have different base rates and the predictor is imperfect, calibration is compatible with at most one equal-error-rate constraint. A team that must show both equal error rates and calibrated risk for each group cannot obtain both from this method, and relaxing to equal opportunity does not by itself keep the original score calibrated.
Resources
Research Papers
Equality of Opportunity in Supervised Learning
We propose a criterion for discrimination against a specified sensitive attribute in supervised learning, where the goal is to predict some target based on available features. Assuming data about the predictor, target, and membership in the protected group are available, we show how to optimally adjust any learned predictor so as to remove discrimination according to our definition. Our framework also improves incentives by shifting the cost of poor classification from disadvantaged groups to the decision maker, who can respond by improving the classification accuracy. In line with other studies, our notion is oblivious: it depends only on the joint statistics of the predictor, the target and the protected attribute, but not on interpretation of individualfeatures. We study the inherent limits of defining and identifying biases based on such oblivious measures, outlining what can and cannot be inferred from different oblivious tests. We illustrate our notion using a case study of FICO credit scores.
Group-Aware Threshold Adaptation for Fair Classification
The fairness in machine learning is getting increasing attention, as its applications in different fields continue to expand and diversify. To mitigate the discriminated model behaviors between different demographic groups, we introduce a novel post-processing method to optimize over multiple fairness constraints through group-aware threshold adaptation. We propose to learn adaptive classification thresholds for each demographic group by optimizing the confusion matrix estimated from the probability distribution of a classification model output. As we only need an estimated probability distribution of model output instead of the classification model structure, our post-processing model can be applied to a wide range of classification models and improve fairness in a model-agnostic manner and ensure privacy. This even allows us to post-process existing fairness methods to further improve the trade-off between accuracy and fairness. Moreover, our model has low computational cost. We provide rigorous theoretical analysis on the convergence of our optimization algorithm and the trade-off between accuracy and fairness of our method. Our method theoretically enables a better upper bound in near optimality than existing method under same condition. Experimental results demonstrate that our method outperforms state-of-the-art methods and obtains the result that is closest to the theoretical accuracy-fairness trade-off boundary.