Reject Option Classification

Description

Reject Option Classification is a post-processing method, proposed by Kamiran and colleagues (2012), that changes some of a trained classifier's decisions to reduce unfair outcomes between groups, without retraining it. It defines a 'critical region' around the decision boundary, where the predicted probability is close to 50 per cent and the model is least sure. Inside that region it gives the favourable outcome to people from the disadvantaged group and the unfavourable outcome to people from the advantaged group, whatever the model predicted, and decisions outside it are left alone. The width of the region is searched on a labelled validation set, so that a chosen fairness measure, such as statistical parity difference or equal opportunity difference, falls within a set bound while accuracy stays as high as possible. The method needs each person's group at prediction time.

Example Use Cases

Fairness

A graduate recruiter's data team has a CV-scoring model whose scores near the shortlisting cut-off are the least certain. Using a labelled validation set, it searches for the narrowest band around that cut-off that brings the difference in shortlisting rates between men and women within five percentage points, and reports how many candidates' outcomes the band changes.

A hospital data science team tests the method on an unplanned-readmission alert model to narrow the gap in missed-readmission rates between insurance groups. It adopts the change only if the gap falls below five percentage points while accuracy drops by under 10 per cent and alert volume changes by under 20 per cent, and drops the method if any limit is missed.

A court service reviewing a pre-trial risk score applies the method with race as the group attribute, so that defendants whose scores sit close to the high-risk cut-off are not decided by small score differences that may reflect the model's bias. The service reports to its oversight body how many decisions the band changes, in which direction, and what it costs in accuracy.

Limitations

Needs a model that outputs scores, because the critical region is defined on them, so a model that outputs only yes/no labels cannot be used. The scores need not be calibrated probabilities, but if they are poorly calibrated or bunched together, the wrong people fall inside the region.
Inside the region the outcome is set by group membership, so the method needs each person's group at decision time and treats people differently by group. This may be unlawful or unacceptable in some settings, such as hiring or lending, and the group attribute must be available when decisions are made.
A person from the advantaged group with a borderline score receives the unfavourable outcome, and one from the other group the favourable outcome, whatever their individual merit. A wider band closes more of the fairness gap but sets more decisions by group instead of by score, so accuracy falls as the band widens.
Cannot address bias in confident predictions outside the region. The method assumes discrimination is concentrated near the decision boundary, so where a model is confidently wrong about one group, the adjustment leaves those decisions as they are.
The band is chosen by grid search on a labelled validation set against a fairness bound (the AIF360 default is plus or minus 0.05). A small or unrepresentative validation set gives an unreliable band, and if no setting meets the bound the search returns a fallback that may not be acceptable.

Resources

Research Papers

Decision Theory for Discrimination-Aware Classification

Faisal Kamiran, Asim Karim, and Xiangliang Zhang•Dec 1, 2012

Social discrimination (e.g., against females) arising from data mining techniques is a growing concern worldwide. In recent years, several methods have been proposed for making classifiers learned over discriminatory data discrimination-aware. However, these methods suffer from two major shortcomings: (1) They require either modifying the discriminatory data or tweaking a specific classification algorithm and (2) They are not flexible w.r.t. discrimination control and multiple sensitive attribute handling. In this paper, we present two solutions for discrimination-aware classification that neither require data modification nor classifier tweaking. Our first and second solutions exploit, respectively, the reject option of probabilistic classifier(s) and the disagreement region of general classifier ensembles to reduce discrimination. We relate both solutions with decision theory for better understanding of the process. Our experiments using real-world datasets demonstrate that our solutions outperform existing state-of-the-art methods, especially at low discrimination which is a significant advantage. The superior performance coupled with flexible control over discrimination and easy applicability to multiple sensitive attributes makes our solutions an important step forward in practical discrimination-aware classification.

Exploiting reject option in classification for social discrimination control

Faisal Kamiran et al.•Jan 1, 2018

Social discrimination is said to occur when an unfavorable decision for an individual is influenced by her membership to certain protected groups such as females and minority ethnic groups. Such discriminatory decisions often exist in historical data. Despite recent works in discrimination-aware data mining, there remains the need for robust, yet easily usable, methods for discrimination control. In this paper, we utilize reject option in classification, a general decision theoretic framework for handling instances whose labels are uncertain, for modeling and controlling discriminatory decisions. Specifically, this framework permits a formal treatment of the intuition that instances close to the decision boundary are more likely to be discriminated in a dataset. Based on this framework, we present three different solutions for discrimination-aware classification. The first solution invokes probabilistic rejection in single or multiple probabilistic classifiers while the second solution relies upon ensemble rejection in classifier ensembles. The third solution integrates one of the first two solutions with situation testing which is a procedure commonly used in the court of law. All solutions are easy to use and provide strong justifications for the decisions. We evaluate our solutions extensively on four real-world datasets and compare their performances with previously proposed discrimination-aware classifiers. The results demonstrate the superiority of our solutions in terms of both performance and flexibility of applicability. In particular, our solutions are effective at removing illegal discrimination from the predictions. (C) 2017 Published by Elsevier Inc.

Software Packages

AIF360

Aug 22, 2018

A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.

Tutorials

AIF360 demo: Reject Option Classification (notebook)

Documentations

aif360.algorithms.postprocessing.RejectOptionClassification ...

Aif360 Developers•Jan 1, 2018

Related Techniques

Name	Description	Assurance Goals
Threshold Optimiser	Threshold Optimiser is a post-processing method, introduced by Hardt, Price and Srebro (2016), that adjusts a trained classifier's decisions to meet a group fairness criterion without retraining it, most often equalised odds (equal true positive and false positive rates in every group) or equal opportunity (equal true positive rates only). It uses a labelled validation set that records each person's group, and traces each group's ROC curve, which shows the true and false positive rates the model's score gives at every possible cut-off. It then chooses a separate cut-off for each group that keeps accuracy, or another chosen objective, as high as the criterion allows. Equalised odds usually cannot be met with one cut-off per group, so each group's decision is randomised between two cut-offs with fixed probabilities. The output is a rule that needs each person's group at prediction time, and if the model gives only yes/no predictions, a small linear optimisation sets the probability of flipping each prediction instead.	Fairness
Calibration with Equality of Opportunity	Calibration with equality of opportunity is a post-processing step that adjusts a classifier's scores so that two groups face the same error rate, in a score-based form of the equal opportunity criterion, while the scores stay calibrated. Calibrated means that, within each group, about 70% of the people given a score of 0.7 really have the outcome. The error rate is computed from the scores rather than from yes-or-no decisions, and for equality of opportunity it is how far, on average, the scores of people who truly have the outcome fall short of 1. Starting from scores already calibrated within each group, the step replaces a randomly chosen share of the scores in the group with the lower error rate by that group's overall rate of the outcome, which makes those predictions less informative until the two error rates match. Calibration is kept at a cost in accuracy for that group, and matching this score-based rate does not guarantee equal true positive rates once a decision threshold is applied.	Fairness Reliability
Multi-Accuracy Boosting	Multi-accuracy boosting is a post-processing method that corrects a trained classifier's systematic errors on subgroups of people, without retraining it. It needs only the classifier's predictions and a small labelled validation set. An auditor, usually a simple model such as a ridge regression or a shallow decision tree, learns to predict where the classifier's predictions are consistently too high or too low, which finds overlapping subgroups without anyone naming them in advance. Each round fits a small correction for the worst subgroup, as in boosting, and the loop stops when the auditor can find no subgroup whose average error exceeds a chosen tolerance. The result is multi-accuracy, meaning unbiased predictions on every subgroup the auditor can identify, which is not the same as equal accuracy across groups, and the audit itself reports where the original model failed.	Fairness