Diverse Counterfactual Explanations
Description
Diverse Counterfactual Explanations (DiCE) explains one decision of a classifier by generating several altered versions of the input, called counterfactuals, each changed just enough that the model would have given the other outcome, and chosen to differ from one another so that a person sees several distinct ways the result could have been different. Where the contrastive explanation method returns, for one input, a minimal set of features that must be present and a minimal set that must stay absent, DiCE returns several complete alternative inputs at once, within limits the user sets on which features may change and over what range. In the original method all the counterfactuals are optimised together by gradient descent on a loss with three parts: a hinge loss that pushes each across the model's decision threshold, a distance term (each continuous feature's change divided by its median absolute deviation in the training data, plus the share of categorical features changed) that keeps each close to the original, and a diversity term, the determinant of a similarity matrix between them, that rewards them for differing. Afterwards, small changes are reset to their original values for as long as the counterfactual still gets the other outcome, to make it sparser, and counterfactuals that break causal rules supplied by the user, such as gaining a degree without growing older, can be filtered out. The method as published needs a differentiable model such as a neural network; the reference library adds random-sampling, genetic-algorithm and nearest-training-example searches that need only the model's predictions.
Example Use Cases
Explainability
A credit-risk validation team at a lender generates five counterfactuals for each of 500 sampled refused applications with every feature free to change, to see what the model treats as enough for approval. Counterfactuals that reach approval by changing only the postcode, or another likely proxy for a protected characteristic, are logged as findings and passed to the fairness testing team.
A hospital team piloting a classifier that places patients with chronic obstructive pulmonary disease in low, moderate or high cardiovascular risk generates counterfactuals restricted to modifiable factors such as blood pressure, cholesterol and smoking. Clinicians check whether the changes the model links to a lower risk category agree with clinical knowledge before the tool is shown to patients.
Transparency
A consumer lender puts up to three counterfactuals in each refusal letter, generated against the model version that made the decision and limited to features the applicant can change (income, balances, loan amount and term), each shown as an application that would have been approved. Before rollout it tests a draft letter with applicants, checking that they can name a change that would have worked and understand that the options are not a promise of approval if they reapply.
Limitations
- A counterfactual is only valid for the model version that produced it. The method assumes the model does not change, and after retraining a previously suggested change can stop flipping the decision, so a person who acts on it may still be refused. Counterfactuals shown to people should be dated, tied to a model version and re-checked whenever the model is updated.
- Feasibility depends on constraints the user supplies. Without them, counterfactuals can change attributes a person cannot change, such as race or sex, or propose impossible combinations: in the original experiments over a third of the counterfactuals that changed education were infeasible, most raising the degree without raising age. Causal rules can only be applied afterwards, as a filter.
- The set returned is not unique. It depends on the weights on distance and diversity, per-feature weights, the number requested and random starting points, and with the library's random-sampling method a different seed gives a different set. The authors note that showing people counterfactuals produced with different settings is a problem, so fix and record settings and seeds.
- Distance is measured feature by feature (scaled change for numbers, changed or not for categories), so it ignores how features move together, treats every category change as equally hard, and can place counterfactuals where there is little real data. In a study of 9,613 patient records, DiCE's counterfactuals were closer and sparser but less plausible than those of a sampling-based alternative.
- The original paper tests no people. Its stand-in for understanding, a nearest-neighbour model trained on the counterfactuals, is in the authors' words a best case of rational extrapolation, and they leave a behavioural study and the cost of choosing among more options to future work. A counterfactual also does not say which features must stay as they are, so a person who raises income by changing job may still be refused. Test the wording with recipients before relying on it.