Visualizing Fit and Lack of Fit in Complex Regression Models with Predictor Effect Plots and Partial Residuals

Predictor effect displays, introduced in this article, visualize the response surface of complex regression models by averaging and conditioning, producing a sequence of 2D line graphs, one graph or set of graphs for each predictor in the regression problem. Partial residual plots visualize lack of fit, traditionally in relatively simple additive regression models. We combine partial residuals with effect displays to visualize both fit and lack of fit simultaneously in complex regression models, plotting residuals from a model around 2D slices of the fitted response surface. Employing fundamental results on partial residual plots along with examples for both real and contrived data, we discuss and illustrate both the strengths and limitations of the resulting graphs. The methods described in this paper are implemented in the effects package for R.

[1]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[2]  C. Davis,et al.  The subject matter of psychology: Volunteers , 1987 .

[3]  J. Nelder.  A Reformulation of Linear Models , 1977 .

[4]  Peter L. Brooks,et al.  Visualizing data , 1997 .

[5]  Richard A. Becker,et al.  The Visual Design and Control of Trellis Display , 1996 .

[6]  Alan Y. Chiang,et al.  Generalized Additive Models: An Introduction With R , 2007, Technometrics.

[7]  Gabor Grothendieck,et al.  Lattice: Multivariate Data Visualization with R , 2008 .

[8]  R. Cook,et al.  Partial Residual Plots in Generalized Linear Models , 1998 .

[9]  Peter O. Pineo,et al.  Occupational Prestige in Canada , 2008 .

[10]  M. Kendall.  Statistical Methods for Research Workers , 1937, Nature.

[11]  R. Dennis Cook,et al.  Exploring Partial Residual Plots , 1993 .

[12]  O. Linton.  Local Regression Models , 2010 .

[13]  Thomas Lumley,et al.  Analysis of Complex Survey Samples , 2004 .

[14]  Drew A. Linzer,et al.  poLCA: An R Package for Polytomous Variable Latent Class Analysis , 2011 .

[15]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[16]  Sanford Weisberg,et al.  An R Companion to Applied Regression , 2010 .

[17]  Russell V. Lenth,et al.  Least-Squares Means: The R Package lsmeans , 2016 .

[18]  M. Braga,et al.  Exploratory Data Analysis , 2018, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[19]  S. Weisberg,et al.  Effect Displays for Linear, Generalized Linear, and Other Models , 2015 .

[20]  John Fox,et al.  Effect Displays for Generalized Linear Models , 1987 .

[21]  C. E. Rogers,et al.  Symbolic Description of Factorial Models for Analysis of Variance , 1973 .

[22]  J. Fox.  Effect Displays in R for Generalised Linear Models , 2003 .

[23]  J. Fox,et al.  A Note on Gender and the Prestige of Occupations , 1989 .

[24]  A. C. Davison,et al.  Statistical models: Name Index , 2003 .

[25]  K. Pearson,et al.  Mathematical contributions to the theory of evolution. IV. On the probable errors of frequency constants and on the influence of random selection on variation and correlation , 1898, Proceedings of the Royal Society of London.

[26]  Issei Fujishiro,et al.  The elements of graphing data , 2005, The Visual Computer.

[27]  Emil Pitkin,et al.  Peeking Inside the Black Box: Visualizing Statistical Learning With Plots of Individual Conditional Expectation , 2013, 1309.6392.