Prejudice Remover Regulariser

Description

The Prejudice Remover Regulariser, proposed by Kamishima and colleagues (2012), trains a classifier with an extra fairness penalty that discourages its predictions from carrying information about a sensitive attribute such as gender or race. The penalty approximates the 'mutual information' between the predicted label and the sensitive attribute, which measures how much knowing one tells you about the other. It is added to the usual training objective, so the model is pushed to fit the labels while keeping that dependence low. The authors built it into logistic regression and describe it as applicable to other classifiers that output probabilities. One parameter sets the penalty strength, so a practitioner can trade some accuracy for a smaller dependence on the sensitive attribute.

Example Use Cases

Fairness

A lender's modelling team retrains its logistic-regression credit-scoring model with the prejudice remover penalty, using applicant gender as the sensitive attribute. It tries several penalty strengths and records how far approval decisions still depend on gender and how much predictive accuracy is lost at each, so the credit committee can choose the strength it will accept.

A health-system analytics team building a postpartum depression risk model on insurance claims trains it with the penalty, with race as the sensitive attribute. It compares disparate impact and equal opportunity difference with an unpenalised model on the same held-out data, to decide whether the penalised model is good enough or another bias-reduction method is needed.

A university building a model to flag students for early support trains it with the penalty so that flags depend less on gender. The team reports to the student-support board how much accuracy the chosen strength costs, and how far the difference in flag rates between genders shrinks, before the model is used.

Limitations

The penalty strength has to be tuned, and no setting is right in advance. In the original experiments on census income data, the strongest penalty tested cut the difference in positive-prediction rates between men and women from 0.19 to 0.01, but accuracy fell from about 85% to 77%. The choice is a judgement about how much accuracy to give up.
The sensitive attribute is an input to the model at training and at prediction time, in the original method and in the AIF360 implementation. The technique cannot be used where the attribute is not collected or cannot be used when decisions are made.
The penalty measures only how much predictions depend on the sensitive attribute, not whether errors fall evenly across groups. Where groups really differ in outcome rates in the data, pushing the dependence down misclassifies some individuals to meet the target, and no equal-error-rate guarantee follows.
The original method and the AIF360 implementation cover logistic regression only. Using the penalty with a neural network or another model means deriving and implementing it yourself, and the optimisation was designed for the logistic form.
Reduced dependence is measured on the training sample. With little data, the model can depend on the sensitive attribute more on new cases than on the training data, which the original paper calls underestimation, so the dependence should be re-measured on held-out data.

Resources

Research Papers

Fairness-Aware Classifier with Prejudice Remover Regularizer

Toshihiro Kamishima et al.•Sep 24, 2012

Software Packages

Fairness-aware Classifier (faclass)

Jan 1, 2012

kamfadm: Fairness-aware Data Mining (tkamishima/kamfadm)

Jul 10, 2023

Fairness-aware Data Mining

AIF360

Aug 22, 2018

A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.

Tutorials

AIF360 tutorial: Medical Expenditure (notebook)

Documentations

Fairness-Aware Machine Learning and Data Mining

Toshihiro Kamishima•Jan 1, 2012

aif360.algorithms.inprocessing.PrejudiceRemover — AIF360 documentation

Related Techniques

Name	Description	Assurance Goals
Adversarial Debiasing	Adversarial debiasing trains a model so that its predictions carry as little information as possible about a protected attribute, such as sex or race. Two neural networks are trained together, a predictor that learns the main task, such as estimating default risk, and an adversary that tries to guess each person's protected attribute from the predictor's output. The predictor is updated to stay accurate while making the adversary's guesses fail, which pushes its outputs towards equal rates of favourable outcomes across groups (demographic parity). If the adversary is also given the true label, the predictor is penalised only for information about the attribute beyond what the label carries, which targets equal error rates across groups (equalised odds). The method needs the protected attribute for every training example and a model trained by gradient descent.	Fairness
Exponentiated Gradient Reduction	Exponentiated gradient reduction trains a classifier that satisfies a chosen fairness criterion, such as demographic parity (equal rates of favourable outcomes across groups) or equalised odds (equal error rates across groups), while keeping error as low as it can. It reduces the problem to a series of ordinary training runs on the same data, each with different example weights. Between runs, a multiplicative update, the exponentiated gradient step, increases the weight on fairness constraints that are currently violated, and a standard classifier is retrained on the reweighted data. The output is a randomised classifier, a weighted mix of the classifiers trained along the way, with one drawn at random for each prediction. The original paper proves that its error and its fairness violation come close to the best achievable, up to a tolerance the user sets and the statistical error from a finite training set.	Fairness
Fair Bayesian Optimisation	Fair Bayesian Optimisation (FairBO) tunes a model's hyperparameters, the settings fixed before training such as tree depth or regularisation strength, to find the most accurate model that also meets a fairness threshold chosen in advance, for example a difference in positive-prediction rates between two groups of at most 0.05. Because it only changes hyperparameters and retrains, it treats the model and its training pipeline as a black box, so it works with any model class and any group fairness metric, can enforce several metrics at once, and returns a model that need not use the protected attribute at prediction time (the attribute is still needed to measure fairness during the search). Each step trains a model with a candidate configuration and measures its validation accuracy and fairness, and two Gaussian processes (statistical models that predict a value together with its uncertainty) are fitted to the results of the configurations tried so far. The next configuration is the one with the highest expected improvement in accuracy weighted by the predicted probability of meeting the fairness constraint; until a fair configuration has been found, the search maximises that probability alone. The output is the best configuration that met the constraint, with its accuracy and fairness measurements.	Fairness