Relabelling
Description
Relabelling, also called data massaging, is a pre-processing method proposed by Kamiran and Calders. It changes some of the outcome labels in a training set so that a protected group and everyone else have the same rate of positive outcomes, such as being hired or approved. A ranking model first scores each example by how likely it is to be positive. The method then promotes the highest-scoring negative examples from the disadvantaged group to positive and demotes the lowest-scoring positive examples from the favoured group to negative, in equal numbers. Because the examples closest to the decision boundary change first and the numbers are equal, the overall share of positive labels stays the same and the number of changes is as small as it can be.
Example Use Cases
Fairness
A public-sector recruiter's ten years of shortlisting records show women shortlisted less often than men with comparable applications. Before training a screening model, the data team flips the labels of the rejected women a ranking model scores highest and the shortlisted men it scores lowest, in equal numbers, until shortlisting rates match and the overall shortlisting rate is unchanged.
A building society finds that past loan decisions approved applicants from one ethnic minority far less often than others with similar finances, and suspects the gap reflects past discretion rather than risk. It flips borderline decisions in both directions, keeping the overall approval rate fixed, and trains its scoring model on the corrected labels while keeping the original labels in a separate file.
Limitations
- It assumes the whole gap in positive rates between groups is discrimination. The number of labels changed is set to bring the gap to zero, even where part of the gap has a legitimate explanation, and the original authors note that such constraints are often too strong.
- It rewrites the record of what happened. The flipped labels no longer match the real decisions or outcomes, so it suits labels that record biased past decisions and is dangerous where labels are measured facts, such as a confirmed diagnosis or a repayment. The authors call the approach intrusive.
- Which records are flipped depends on a ranking model trained on the same biased data. A poor ranker flips the wrong people, and in the original experiments the discrimination and accuracy of the final classifier varied with the learner trained afterwards.
- It handles one binary protected attribute and a binary outcome. Gender and ethnicity together, or a numeric attribute such as income, are not covered, so correcting one attribute can leave intersectional gaps.
- Accuracy falls as discrimination falls, and the authors prove the trade-off cannot be avoided. Discrimination is removed from the training data, but a model trained on it can still show some on new data.
Resources
Research Papers
Data preprocessing techniques for classification without discrimination
Recently, the following Discrimination-Aware Classification Problem was introduced: Suppose we are given training data that exhibit unlawful discrimination; e.g., toward sensitive attributes such as gender or ethnicity. The task is to learn a classifier that optimizes accuracy, but does not have this discrimination in its predictions on test data. This problem is relevant in many settings, such as when the data are generated by a biased decision process or when the sensitive attribute serves as a proxy for unobserved features. In this paper, we concentrate on the case with only one binary sensitive attribute and a two-class classification problem. We first study the theoretically optimal trade-off between accuracy and non-discrimination for pure classifiers. Then, we look at algorithmic solutions that preprocess the data to remove discrimination before a classifier is learned. We survey and extend our existing data preprocessing techniques, being suppression of the sensitive attribute, massaging the dataset by changing class labels, and reweighing or resampling the data to remove discrimination without relabeling instances. These preprocessing techniques have been implemented in a modified version of Weka and we present the results of experiments on real-life data.
Classifying without discriminating
Classification models usually make predictions on the basis of training data. If the training data is biased towards certain groups or classes of objects, e.g., there is racial discrimination towards black people, the learned model will also show discriminatory behavior towards that particular community. This partial attitude of the learned model may lead to biased outcomes when labeling future unlabeled data objects. Often, however, impartial classification results are desired or even required by law for future data objects in spite of having biased training data. In this paper, we tackle this problem by introducing a new classification scheme for learning unbiased models on biased training data. Our method is based on massaging the dataset by making the least intrusive modifications which lead to an unbiased dataset. On this modified dataset we then learn a non-discriminating classifier. The proposed method has been implemented and experimental results on a credit approval dataset show promising results: in all experiments our method is able to reduce the prejudicial behavior for future classification significantly without loosing too much predictive accuracy.