Taylor Decomposition

Description

Taylor Decomposition explains one prediction of a neural network by splitting its output score into a contribution from each input, such as each pixel. It starts from a reference input, called a root point, that is close to the real input but for which the network's score is zero. Each input's contribution is the slope of the score with respect to that input at the root point, multiplied by how far the input sits from the root, and terms that involve several inputs at once are left out. Finding a good root point is slow for large networks, so the deep variant applies the same idea to one neuron at a time and passes the contributions backwards through the layers, which gives rules close to those of Layer-wise Relevance Propagation. The result is a relevance score for every input that can be drawn as a heatmap.

Example Use Cases

Explainability

A quality team uses a convolutional network to flag defective printed circuit boards. For each flagged board they draw the deep Taylor heatmap and check that the highlighted pixels sit on the defect and not on a board label, the edge or the background, before they trust the flags enough to stop manual inspection.

A hospital's model-validation group audits a ReLU network that classifies chest X-rays. For a sample of positive cases, deep Taylor relevance shows whether the evidence for the finding lies inside the lungs or on markers, text or borders added by the imaging equipment, before the group signs the model off for a pilot.

A bank's data science team has a ReLU network that scores small-business loan applications from tabular inputs. For a declined application, deep Taylor relevance lists which inputs supplied most of the score, so a reviewer can see whether the decision rests on repayment history and income or on a field such as postcode.

Limitations

The explanation depends on the choice of root point, and different reference inputs give different attributions. Finding the nearest root needs an iterative optimisation that is slow for large networks and may not converge, which is why the source applies the deep variant to large image networks.
Plain Taylor decomposition uses only first-order terms, so joint effects of several inputs stay unattributed and the scores need not add up to the prediction. In the source's tests the nearest-root version lost a large share of the relevance, so its heatmaps can leave much of the prediction unexplained.
The deep variant's guarantees, scores that add up to the prediction and are never negative, are proved for ReLU-style layers with zero or negative biases, and iNNvestigate documents support for ReLU networks only. Other activations, positive biases or normalisation layers need workarounds that the proofs do not cover.
Heatmaps explain one input at a time and are hard to validate. The source judges them by eye and by checking that relevance is conserved on MNIST digits and ImageNet photos, so a plausible-looking map does not prove that the model relies on the highlighted inputs.
Requires access to internal layer operations and supported redistribution rules. A prediction-only service cannot supply this layer-wise decomposition, and an image-derived rule should not be assumed valid for arbitrary text or attention models.

Resources

Research Papers

Explaining NonLinear Classification Decisions with Deep Taylor Decomposition

Grégoire Montavon et al.•Dec 8, 2015

Nonlinear methods such as Deep Neural Networks (DNNs) are the gold standard for various challenging machine learning problems, e.g., image classification, natural language processing or human action recognition. Although these methods perform impressively well, they have a significant disadvantage, the lack of transparency, limiting the interpretability of the solution and thus the scope of application in practice. Especially DNNs act as black boxes due to their multilayer nonlinear structure. In this paper we introduce a novel methodology for interpreting generic multilayer neural networks by decomposing the network classification decision into contributions of its input elements. Although our focus is on image classification, the method is applicable to a broad set of input data, learning tasks and network architectures. Our method is based on deep Taylor decomposition and efficiently utilizes the structure of the network by backpropagating the explanations from the output to the input layer. We evaluate the proposed method empirically on the MNIST and ILSVRC data sets.

Software Packages

lrp_toolbox

May 26, 2016

The LRP Toolbox provides simple and accessible stand-alone implementations of LRP for artificial neural networks supporting Matlab and Python. The Toolbox realizes LRP functionality for the Caffe Deep Learning Framework as an extension of Caffe source code published in 10/2015.

iNNvestigate (albermax/innvestigate)

Oct 8, 2026

A toolbox to iNNvestigate neural networks' predictions!

Tutorials

Methods for interpreting and understanding deep neural networks

Grégoire Montavon, Wojciech Samek, and Klaus-Robert Müller•Feb 1, 2018

Documentations

Analyzer — iNNvestigate documentation

Related Techniques

Name	Description	Assurance Goals
Layer-wise Relevance Propagation	Layer-wise Relevance Propagation (LRP) explains a neural network's prediction by working backwards from the output to the inputs. It treats the prediction score as a quantity called relevance, then passes that relevance back through each layer in proportion to how much each neuron contributed in the forward pass. The rules for this redistribution are chosen to keep the total relevance roughly constant from layer to layer, so the input scores add up to the prediction apart from small amounts absorbed by bias terms and stabilisers. Different rules can be used for different types of layer. The result is a relevance score for each input, such as a pixel or a feature, with positive values for inputs that supported the prediction and negative values for inputs that counted against it.	Explainability Transparency
DeepLIFT	DeepLIFT (Deep Learning Important FeaTures) explains a single prediction of a neural network by dividing the difference between the model's output and its output on a reference input among the input features. The reference is a neutral input chosen for the problem, such as an all-zero image or a blurred copy of the input. The method measures how far each neuron's activation is from its value on the reference, then passes those differences backwards through the layers in one pass, applying a rule for each layer type. A plain gradient (the rate of change of the output with respect to an input) is zero where the output has levelled off, but DeepLIFT can still assign credit there. The scores add up to the output difference and mean something only relative to the reference chosen.	Explainability Transparency
Contextual Decomposition	Contextual Decomposition explains why an LSTM, a neural network that reads text one word at a time, made a particular prediction. The user picks a word or phrase, and the method splits the network's output score into two parts: the part due to that phrase alone, and the part due to everything else, including how the phrase interacts with the words around it. Comparing a phrase's score with the scores of its parts reveals effects that word-by-word methods miss, such as a negation reversing the sentiment of the word that follows it. The split is tracked through the LSTM's internal gates, so the model itself is not changed or retrained. The output is a numerical score for each phrase examined.	Explainability Transparency