Explainability
Representation Analysis
Techniques analyzing internal model representations
3 subcategories • 11 techniques
Concept Identification
Identifies learned concepts in models (e.g., CAVs, Neuron Activation Analysis)
Decomposition
Breaks down predictions into components (e.g., Taylor Decomposition, Contextual Decomposition)
Dimensionality Reduction
Reduces complexity for understanding (e.g., PCA, t-SNE, UMAP)
Techniques
11 techniques
| Goals | Models | Data Types | Description | |||
|---|---|---|---|---|---|---|
| DeepLIFT | Algorithmic | Requirements/model Internals Architecture/neural Networks +2 | Any | DeepLIFT (Deep Learning Important FeaTures) explains a single prediction of a neural network by dividing the difference... | ||
| Layer-wise Relevance Propagation | Algorithmic | Requirements/model Internals Architecture/neural Networks +3 | Any | Layer-wise Relevance Propagation (LRP) explains a neural network's prediction by working backwards from the output to... | ||
| Contextual Decomposition | Algorithmic | Architecture/neural Networks/recurrent Requirements/white Box +1 | Text | Contextual Decomposition explains why an LSTM, a neural network that reads text one word at a time, made a particular... | ||
| Taylor Decomposition | Algorithmic | Requirements/model Internals Architecture/neural Networks +3 | Any | Taylor Decomposition explains one prediction of a neural network by splitting its output score into a contribution from... | ||
| Factor Analysis | Algorithmic | Architecture/model Agnostic Paradigm/unsupervised +1 | Tabular | Factor analysis looks for a small number of hidden factors that explain why a larger set of measured variables, such as... | ||
| Principal Component Analysis | Algorithmic | Architecture/model Agnostic Paradigm/unsupervised +1 | Any | Principal component analysis (PCA) summarises a dataset that has many numeric features by finding a few new axes along... | ||
| t-SNE | Algorithmic | Architecture/model Agnostic Requirements/black Box | Any | t-SNE (t-distributed Stochastic Neighbour Embedding) turns high-dimensional data into a two- or three-dimensional map... | ||
| UMAP | Algorithmic | Architecture/model Agnostic Requirements/black Box | Any | UMAP (Uniform Manifold Approximation and Projection) is a non-linear dimensionality reduction method that turns... | ||
| Neuron Activation Analysis | Algorithmic | Architecture/neural Networks Requirements/model Internals +1 | Text | Neuron activation analysis examines when individual neurons in a neural network respond strongly. The analyst runs a set... | ||
| Concept Activation Vectors | Algorithmic | Architecture/neural Networks Requirements/gradient Access +2 | Any | A concept activation vector (CAV) is a direction inside a neural network that stands for a human idea such as 'stripes'.... | ||
| Sparse Autoencoder Feature Analysis | Algorithmic | Architecture/neural Networks/transformer Architecture/neural Networks/transformer/llm +2 | Text | Sparse autoencoder feature analysis looks for human-interpretable units inside a neural network, usually a language... |
Rows per page
Page 1 of 1