Coefficient Magnitudes (in Linear Models)

Description

Coefficient Magnitudes assess feature influence in linear models by examining the absolute values of their coefficients. Features with larger absolute coefficients are considered to have a stronger impact on the prediction, while the sign of the coefficient indicates the direction of that influence (positive or negative). This technique provides a straightforward and transparent way to understand the direct linear relationship between each input feature and the model's output.

Example Use Cases

Explainability

Interpreting which features influence housing price predictions in linear regression, such as identifying that 'number of bedrooms' has a larger positive impact than 'distance to city centre' based on coefficient magnitudes.

Transparency

Explaining the factors contributing to customer lifetime value (CLV) in a linear model, showing how 'average monthly spend' has a strong positive coefficient, making the model transparent for business stakeholders.

Limitations

Only valid for linear relationships; it cannot capture complex non-linear patterns or interactions between features.
Highly sensitive to feature scaling; features with larger numerical ranges can appear more important even if their true impact is smaller.
Can be misleading in the presence of multicollinearity, where correlated features may split importance or have unstable coefficients.
Does not imply causation; a strong correlation (large coefficient) does not necessarily mean a causal relationship.

Resources

Research Papers

The Elements of Statistical Learning

Jan 1, 2009

Software Packages

statsmodels: Econometric and statistical modeling

Jan 1, 2009

Tutorials

Interpreting Linear Regression Coefficients

Jim Frost•Apr 12, 2017

Documentations

scikit-learn Linear Models

Scikit-learn Developers•Jan 1, 2007

Related Techniques

Name	Description	Assurance Goals
SHapley Additive exPlanations	SHAP explains model predictions by quantifying how much each input feature contributes to the outcome. It assigns an importance score to every feature, indicating whether it pushes the prediction towards or away from the average. The method systematically evaluates how predictions change as features are included or excluded, drawing on game theory concepts to ensure a fair distribution of contributions.	Explainability Fairness Reliability
Permutation Importance	Permutation Importance quantifies a feature's contribution to a model's performance by randomly shuffling its values and measuring the resulting drop in predictive accuracy. If shuffling a feature significantly degrades the model's performance, that feature is considered important. This model-agnostic technique helps identify which inputs are genuinely driving predictions, rather than just being correlated with the outcome.	Explainability Reliability
Mean Decrease Impurity	Mean Decrease Impurity (MDI) quantifies a feature's importance in tree-based models (e.g., Random Forests, Gradient Boosting Machines) by measuring the total reduction in impurity (e.g., Gini impurity, entropy) across all splits where the feature is used. Features that lead to larger, more consistent reductions in impurity are considered more important, indicating their effectiveness in creating homogeneous child nodes and improving predictive accuracy. This makes MDI a computationally efficient method for feature selection and model validation in tree-based ensembles.	Explainability Reliability
Generalized Additive Models	An intrinsically interpretable modelling technique that extends linear models by allowing flexible, nonlinear relationships between individual features and the target whilst maintaining the additive structure that preserves transparency. Each feature's effect is modelled separately as a smooth function, visualised as a curve showing how the feature influences predictions across its range. GAMs achieve this through spline functions or other smoothing techniques that capture complex patterns in individual variables without interactions, making them particularly valuable for domains requiring both predictive accuracy and model interpretability.	Transparency Explainability Reliability