Concept Activation Vectors
Description
A concept activation vector (CAV) is a direction inside a neural network that stands for a human idea such as 'stripes'. The analyst supplies example inputs showing the concept and random counter-examples, and trains a simple linear classifier to separate the network's activations (the values a chosen layer produces) for the two sets. The CAV is the direction at right angles to that classifier's dividing boundary, pointing towards the concept examples. Testing with CAVs (TCAV) then reports, for one output class, the share of its inputs whose class score would rise if the activations moved in the concept's direction, and repeating this with many random counter-example sets and applying a significance test guards against a meaningless direction. Concept sets built from examples of a protected characteristic, such as gender or ethnicity, turn the test into a check of whether a class is sensitive to that characteristic, as the original paper demonstrates.
Example Use Cases
Explainability
Auditing a medical imaging model to check whether its chest X-ray classifications are sensitive to diagnostic concepts (like 'tumour characteristics') or to irrelevant ones (like 'scanner type' or 'patient positioning'), so the clinical team can judge whether the model relies on medically relevant information.
Fairness
A team auditing an image classifier used for content tagging builds concept sets from photographs of people of different ethnicities and genders, with random photographs as the comparison, and tests each output class for significant sensitivity to those concepts, as the original paper did. Classes with significant sensitivity to an ethnicity or gender concept are listed for retraining with balanced data; a low score is not taken as clearance.
Transparency
A developer of a driver-assistance system documents, for a type-approval authority, how its traffic-sign classifier responds to visual concepts. The report gives significance-tested TCAV scores for the 'stop sign' class against concepts such as graffiti, snow cover and low sun glare, so the authority can see what the classifier's predictions are sensitive to.
Limitations
- Requires clearly defined concept examples and non-concept examples, which can be hard to obtain for abstract or subjective concepts. If the examples differ from the random counter-examples in other ways, such as image source or lighting, the CAV mixes those differences in with the concept.
- Assumes the concept is represented as a direction that a linear classifier can pick out at the chosen layer. If the concept is not linearly separable there, the CAV is unreliable, so check that the classifier separates the two sets well before reading any scores.
- Results depend heavily on which layer is examined, because layers capture different levels of abstraction. A concept can be invisible at one layer and strong at another, so a single-layer result should not be generalised.
- The significance test trains hundreds of classifiers per concept and layer (the paper uses about 500), so cost multiplies with the number of concepts and layers tested. FastCAV, a 2025 preprint, reports speed-ups of up to 63.6 times.
- Scores are class-level and describe sensitivity, not cause. A TCAV score is the share of a class's inputs whose score rises along the concept direction, so it does not explain an individual decision, and a high score does not show that the concept causes the prediction.
Resources
Research Papers
FastCAV: Efficient Computation of Concept Activation Vectors for Explaining Deep Neural Networks
Concepts such as objects, patterns, and shapes are how humans understand the world. Building on this intuition, concept-based explainability methods aim to study representations learned by deep neural networks in relation to human-understandable concepts. Here, Concept Activation Vectors (CAVs) are an important tool and can identify whether a model learned a concept or not. However, the computational cost and time requirements of existing CAV computation pose a significant challenge, particularly in large-scale, high-dimensional architectures. To address this limitation, we introduce FastCAV, a novel approach that accelerates the extraction of CAVs by up to 63.6x (on average 46.4x). We provide a theoretical foundation for our approach and give concrete assumptions under which it is equivalent to established SVM-based methods. Our empirical results demonstrate that CAVs calculated with FastCAV maintain similar performance while being more efficient and stable. In downstream applications, i.e., concept-based explanation methods, we show that FastCAV can act as a replacement leading to equivalent insights. Hence, our approach enables previously infeasible investigations of deep models, which we demonstrate by tracking the evolution of concepts during model training.
Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
The interpretation of deep learning models is a challenge due to their size, complexity, and often opaque internal state. In addition, many systems, such as image classifiers, operate on low-level features rather than high-level concepts. To address these challenges, we introduce Concept Activation Vectors (CAVs), which provide an interpretation of a neural net’s internal state in terms of human-friendly concepts. The key idea is to view the high-dimensional internal state of a neural net as an aid, not an obstacle. We show how to use CAVs as part of a technique, Testing with CAVs (TCAV), that uses directional derivatives to quantify the degree to which a user-defined concept is important to a classification result–for example, how sensitive a prediction of “zebra” is to the presence of stripes. Using the domain of image classification as a testing ground, we describe how CAVs may be used to explore hypotheses and generate insights for a standard image classification network as well as a medical application.