Adversarial Debiasing
Description
Adversarial debiasing trains a model so that its predictions carry as little information as possible about a protected attribute, such as sex or race. Two neural networks are trained together, a predictor that learns the main task, such as estimating default risk, and an adversary that tries to guess each person's protected attribute from the predictor's output. The predictor is updated to stay accurate while making the adversary's guesses fail, which pushes its outputs towards equal rates of favourable outcomes across groups (demographic parity). If the adversary is also given the true label, the predictor is penalised only for information about the attribute beyond what the label carries, which targets equal error rates across groups (equalised odds). The method needs the protected attribute for every training example and a model trained by gradient descent.
Example Use Cases
Fairness
A technology company trains a CV-screening model with an adversary that tries to guess each applicant's gender from the model's shortlisting score, so that selection rates come out similar for men and women (demographic parity) instead of leaning on CV features that happen to correlate with gender.
A lender trains a default-risk model for loan approvals with an adversary that is given each applicant's score and true repayment outcome and tries to guess their age band, so that false-positive and false-negative rates are similar across age bands (equalised odds) while predictive accuracy is kept.
A hospital group trains a COVID-19 screening model on emergency-department data with an adversary that tries to guess patient ethnicity from the model's score and the true test result, so that error rates are similar across ethnic groups while screening performance stays clinically useful.
Limitations
- The adversary only sees what it is given, usually the predictor's score and, for equalised odds, the true label. The method therefore targets one output-level criterion and gives no assurance that hidden layers are free of protected-attribute information, or that a different criterion holds.
- Needs the protected attribute for every training example, so it cannot be used where that attribute was never collected or may not be held.
- Demographic parity and equalised odds generally cannot both be met, so one must be chosen before training. Equal error rates can also come from worsening one group's rates: in the original Adult income experiment, the false-positive rate for women rose from 0.025 to 0.065.
- Training two networks against each other is unstable. The weight given to the adversary has to be tuned (adversary_loss_weight in AIF360), and a wrong setting can make training diverge or leave the result closer to a different criterion than the one intended.
- The paper's guarantee that outputs meet the chosen criterion holds only if both networks converge and the adversary is strong enough. A weak adversary leaves bias in place, and the convexity assumptions behind the proof do not hold for typical neural networks, so fairness has to be checked on held-out data after training.
Resources
Research Papers
On the Fairness ROAD: Robust Optimization for Adversarial Debiasing
In the field of algorithmic fairness, significant attention has been put on group fairness criteria, such as Demographic Parity and Equalized Odds. Nevertheless, these objectives, measured as global averages, have raised concerns about persistent local disparities between sensitive groups. In this work, we address the problem of local fairness, which ensures that the predictor is unbiased not only in terms of expectations over the whole population, but also within any subregion of the feature space, unknown at training time. To enforce this objective, we introduce ROAD, a novel approach that leverages the Distributionally Robust Optimization (DRO) framework within a fair adversarial learning objective, where an adversary tries to infer the sensitive attribute from the predictions. Using an instance-level re-weighting strategy, ROAD is designed to prioritize inputs that are likely to be locally unfair, i.e. where the adversary faces the least difficulty in reconstructing the sensitive attribute. Numerical experiments demonstrate the effectiveness of our method: it achieves Pareto dominance with respect to local fairness and accuracy for a given global fairness level across three standard datasets, and also enhances fairness generalization under distribution shift.
Mitigating Unwanted Biases with Adversarial Learning
Machine learning is a tool for building models that accurately represent input training data. When undesired biases concerning demographic groups are in the training data, well-trained models will reflect those biases. We present a framework for mitigating such biases by including a variable for the group of interest and simultaneously learning a predictor and an adversary. The input to the network X, here text or census data, produces a prediction Y, such as an analogy completion or income bracket, while the adversary tries to model a protected variable Z, here gender or zip code. The objective is to maximize the predictor's ability to predict Y while minimizing the adversary's ability to predict Z. Applied to analogy completion, this method results in accurate predictions that exhibit less evidence of stereotyping Z. When applied to a classification task using the UCI Adult (Census) Dataset, it results in a predictive model that does not lose much accuracy while achieving very close to equality of odds (Hardt, et al., 2016). The method is flexible and applicable to multiple definitions of fairness as well as a wide range of gradient-based learning models, including both regression and classification tasks.