Contrastive Explanation Method
Description
The Contrastive Explanation Method (CEM) explains one prediction of a classifier, such as a neural network, by describing what is present and what is absent in the input. A pertinent positive is the smallest set of features already present that is enough for the model to give the same prediction. A pertinent negative is the smallest set of features not currently present that would change the prediction to a different class if they were added. CEM finds both by solving an optimisation problem that searches for the smallest change to the input, and it can use an autoencoder (a model trained to reconstruct realistic inputs) to keep the changed inputs realistic. An explanation reads like 'this digit is a 3 because these strokes are present and because a short bar at the top is absent', since adding that bar would make it a 5.
Example Use Cases
Explainability
A procurement team uses a neural-network model that rates invoices as low, medium or high risk. For a high-risk invoice, the pertinent positives are the few triggered events that are enough for that rating, such as a large amount and a vendor with only a PO box. The pertinent negatives are events that did not occur but would lower the rating if they had, such as the vendor being registered. Reviewers check both before blocking payment.
A clinician reviews a decision-support model that suggests flu rather than pneumonia for a patient. The pertinent positives are the findings that are enough for that suggestion, such as cough, cold and fever. The pertinent negatives are findings that are absent but would move the suggestion to pneumonia if present, such as sputum and chills. The clinician compares both with their own reasoning to judge whether the model weighs the right findings.
Transparency
A council uses a model to screen applications for housing support. When an application is declined, the pertinent negatives show which missing items, such as proof of tenancy, would have changed the outcome if present. A caseworker checks that each item is something the applicant can provide, then includes it in the decision notice.
Limitations
- Each explanation needs its own optimisation run for a single instance, so explaining many decisions is slow. Models whose gradients cannot be accessed need numerical gradient estimates, which slows it further.
- Several settings shape the result, including how strongly the search favours small changes, how confident the model must be in the new class, and whether an autoencoder is used. Different settings can give different pertinent positives and negatives for the same instance, so they need checking with someone who knows the domain.
- Changed inputs can be unrealistic. Without an autoencoder trained on good-quality data, a pertinent negative may describe an input that could never occur, so the explanation says little about real cases. Where data is scarce or poor, a good autoencoder may not be possible.
- Absence has to be defined by the user. CEM adds or removes features relative to a value that stands for no signal, such as a blank pixel, and a poor choice gives meaningless pertinent negatives. This is natural for images and binary indicators, and needs domain judgement for continuous or categorical tabular data.
- A pertinent negative is the smallest change the model responds to, not advice a person can act on. It may involve a feature that cannot be changed, such as age, and nothing in the method checks that the change is feasible. It is most useful when the two classes are close; if they are very different, there can be too many pertinent negatives to read.
Resources
Research Papers
Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives
In this paper we propose a novel method that provides contrastive explanations justifying the classification of an input by a black box classifier such as a deep neural network. Given an input we find what should be %necessarily and minimally and sufficiently present (viz. important object pixels in an image) to justify its classification and analogously what should be minimally and necessarily \emph{absent} (viz. certain background pixels). We argue that such explanations are natural for humans and are used commonly in domains such as health care and criminology. What is minimally but critically \emph{absent} is an important part of an explanation, which to the best of our knowledge, has not been explicitly identified by current explanation methods that explain predictions of neural networks. We validate our approach on three real datasets obtained from diverse domains; namely, a handwritten digits dataset MNIST, a large procurement fraud dataset and a brain activity strength dataset. In all three cases, we witness the power of our approach in generating precise explanations that are also easy for human experts to understand and evaluate.