Generalized Additive Models
Description
A generalised additive model (GAM) builds its prediction by adding a baseline and one smooth curve for each input feature. It extends linear and logistic regression by letting each feature's effect bend, so the model can follow patterns such as risk that climbs steeply after a certain age. Each curve is learned from the data with a smoothing method, such as splines (flexible curves built from joined polynomial pieces). Plotting the curves shows how each feature moves the prediction across its range, so the model can be read directly without a separate explanation method. Because the curves are added together, the model does not capture the joint effect of two features unless a combined term is added by hand.
Example Use Cases
Transparency
A hospital builds its 30-day readmission model as a GAM so that its governance committee can audit it. The committee reviews one curve each for age, blood pressure and medication adherence, and queries any stretch where predicted risk falls as a known risk factor rises, since that can point to a data problem rather than a real effect.
Explainability
A lender uses a GAM for credit scoring so that it can tell a declined applicant how income, credit history and debt-to-income ratio each moved their score. Each factor has its own curve, so the explanation comes straight from the model and needs no separate explanation tool.
An environmental agency models river nitrate levels with a GAM and plots the curve for rainfall. The curve shows the effect levelling off above a certain rainfall, which analysts report to catchment managers instead of a single regression coefficient.
Limitations
- Each feature acts separately, so the model cannot represent effects that depend on two features together, such as the impact of a dose varying with age. Product terms can be added by hand, but each must be thought of in advance and adds a surface that is harder to read than a curve.
- The smoothness of each curve is a modelling choice. Cross-validation or penalised fitting selects it automatically, but too little smoothing gives wiggly curves that look like real effects and too much hides genuine bends, so each curve should be read with its confidence band.
- When features are strongly related, such as income and age, a nonlinear form of collinearity called concurvity lets one curve absorb part of another's effect. The individual curves can then misstate how much each feature influences the prediction.
- A model with dozens of smooth terms produces dozens of curves. Reading them all is slow, so reviewers tend to inspect only the largest effects, and the transparency benefit shrinks as the number of terms grows.
- Terms add on the chosen link scale. In a logistic GAM they are contributions to log-odds, not independent additions to readmission probability or guaranteed causal feature effects.
Resources
Research Papers
Generalized Additive Models
Likelihood-based regression models such as the normal linear regression model and the linear logistic model, assume a linear (or some other parametric) form for the covariates X1, X2, …, Xp. We introduce the class of generalized additive models which replaces the linear form Σ βjXj by a sum of smooth functions Σ sj(Xj). The sj(·)'s are unspecified functions that are estimated using a scatterplot smoother, in an iterative procedure we call the local scoring algorithm. The technique is applicable to any likelihood-based regression model: the class of generalized linear models contains many of these. In this class the linear predictor η = Σ βjXj is replaced by the additive predictor Σ sj(Xj); hence, the name generalized additive models. We illustrate the technique with binary response and survival data. In both cases, the method proves to be useful in uncovering nonlinear covariate effects. It has the advantage of being completely automatic, i.e., no "detective work" is needed on the part of the statistician. As a theoretical underpinning, the technique is viewed as an empirical method of maximizing the expected log likelihood, or equivalently, of minimizing the Kullback-Leibler distance to the true model.
Fast Stable Restricted Maximum Likelihood and Marginal Likelihood Estimation of Semiparametric Generalized Linear Models
Recent work by Reiss and Ogden provides a theoretical basis for sometimes preferring restricted maximum likelihood (REML) to generalized cross-validation (GCV) for smoothing parameter selection in semiparametric regression. However, existing REML or marginal likelihood (ML) based methods for semiparametric generalized linear models (GLMs) use iterative REML or ML estimation of the smoothing parameters of working linear approximations to the GLM. Such indirect schemes need not converge and fail to do so in a non-negligible proportion of practical analyses. By contrast, very reliable prediction error criteria smoothing parameter selection methods are available, based on direct optimization of GCV, or related criteria, for the GLM itself. Since such methods directly optimize properly defined functions of the smoothing parameters, they have much more reliable convergence properties. The paper develops the first such method for REML or ML estimation of smoothing parameters. A Laplace approximation is used to obtain an approximate REML or ML for any GLM, which is suitable for efficient direct optimization. This REML or ML criterion requires that Newton–Raphson iteration, rather than Fisher scoring, be used for GLM fitting, and a computationally stable approach to this is proposed. The REML or ML criterion itself is optimized by a Newton method, with the derivatives required obtained by a mixture of implicit differentiation and direct methods. The method will cope with numerical rank deficiency in the fitted model and in fact provides a slight improvement in numerical robustness on the earlier method of Wood for prediction error criteria based smoothness selection. Simulation results suggest that the new REML and ML methods offer some improvement in mean-square error performance relative to GCV or Akaike’s information criterion in most cases, without the small number of severe undersmoothing failures to which Akaike’s information criterion and GCV are prone. This is achieved at the same computational cost as GCV or Akaike’s information criterion. The new approach also eliminates the convergence failures of previous REML- or ML-based approaches for penalized GLMs and usually has lower computational cost than these alternatives. Example applications are presented in adaptive smoothing, scalar on function regression and generalized additive model selection.