Partial Dependence Plots

Description

Partial Dependence Plots show how changing one or two features affects a model's predictions on average. The technique works by varying the selected feature(s) across their full range whilst keeping all other features fixed at their original values, then averaging the predictions. This creates a clear visualisation of whether increasing or decreasing a feature tends to increase or decrease predictions, and reveals patterns like linear trends, plateaus, or threshold effects that help explain model behaviour.

Example Use Cases

Explainability

Analysing how house prices change with property size in a real estate prediction model, revealing whether the relationship is linear or if there are diminishing returns for very large properties.

Examining how customer age affects predicted loan default probability in a credit scoring model, showing whether risk increases steadily with age or has specific age ranges with higher risk.

Visualising how temperature affects crop yield predictions in agricultural models, identifying optimal temperature ranges and potential threshold effects.

Limitations

Assumes features are independent when averaging, which can be misleading when features are highly correlated.
Shows only average effects across all instances, potentially hiding important variations in how different subgroups respond to feature changes.
Cannot reveal instance-specific effects or interactions between the plotted feature and other features.
May be computationally expensive for large datasets since it requires making predictions across the full range of feature values.

Resources

DanielKerrigan/PDPilot

Software Package

Peeking Inside the Black Box: Visualizing Statistical Learning with Plots of Individual Conditional Expectation

Research Paper•Alex Goldstein et al.•Sep 25, 2013

SauceCat/PDPbox

Software Package

iPDP: On Partial Dependence Plots in Dynamic Modeling Scenarios

Research Paper•Maximilian Muschalik et al.•Jun 13, 2023

How to Interpret Models: PDP and ICE | Towards Data Science

Tutorial

Related Techniques

Name	Description	Assurance Goals
Ridge Regression Surrogates	This technique approximates a complex model by training a ridge regression (a linear model with L2 regularization) on the original model's predictions. The ridge regression serves as a global surrogate that balances fidelity and interpretability, capturing the main linear relationships that the complex model learned while ignoring noise due to regularization.	Explainability Transparency
Local Interpretable Model-Agnostic Explanations	LIME (Local Interpretable Model-agnostic Explanations) explains individual predictions by approximating the complex model's behaviour in a small neighbourhood around a specific instance. It works by creating perturbed versions of the input (e.g., removing words from text, changing pixel values in images, or varying feature values), obtaining the model's predictions for these variations, and training a simple interpretable model (typically linear regression) weighted by proximity to the original instance. The coefficients of this local surrogate model reveal which features most influenced the specific prediction.	Explainability Transparency
SHapley Additive exPlanations	SHAP explains model predictions by quantifying how much each input feature contributes to the outcome. It assigns an importance score to every feature, indicating whether it pushes the prediction towards or away from the average. The method systematically evaluates how predictions change as features are included or excluded, drawing on game theory concepts to ensure a fair distribution of contributions.	Explainability Fairness Reliability
Coefficient Magnitudes (in Linear Models)	Coefficient Magnitudes assess feature influence in linear models by examining the absolute values of their coefficients. Features with larger absolute coefficients are considered to have a stronger impact on the prediction, while the sign of the coefficient indicates the direction of that influence (positive or negative). This technique provides a straightforward and transparent way to understand the direct linear relationship between each input feature and the model's output.	Explainability Transparency
t-SNE	t-SNE (t-Distributed Stochastic Neighbour Embedding) is a non-linear dimensionality reduction technique that creates 2D or 3D visualisations of high-dimensional data by preserving local neighbourhood relationships. The algorithm converts similarities between data points into joint probabilities in the high-dimensional space, then tries to minimise the divergence between these probabilities and those in the low-dimensional embedding. This approach excels at revealing cluster structures and local patterns, making it particularly effective for exploratory data analysis and understanding complex data relationships that linear methods like PCA might miss.	Explainability
RuleFit	RuleFit is a method that creates an interpretable model by combining linear terms with decision rules. It first extracts potential rules from ensemble trees, then builds a sparse linear model where those rules (binary conditions) and original features are used as predictors, with regularization to keep the model simple. The final model is a linear combination of a small set of rules and original features, balancing interpretability with predictive power.	Explainability Transparency