A Review of Robust Regression and Diagnostic Procedures in Linear Regression

The concern over outliers is old since Bernoulli (see [12]), reviewed historically by [11] and updated with [10] in their encyclopedia textbook. James et al.[46] used simulation technique to compare some recent published outlier detection procedures.The history of adept and diagnosis of outliers is traced from old and presence comments. Theil-type or Rank, Brown-Mood, Lp, M, adaptive M, GM, and Trimmed-Winsorization estimators are the most popular estimators that we will review in this paper as an application to outlier accommodation. We will review and compare the most numerical and graphical displays based on residuals to flag outliers.

[1]  P. Prescott.  An Approximate Test for Outliers in Linear Models , 1975 .

[2]  F. J. Anscombe,et al.  Rejection of Outliers , 1960 .

[3]  Daniel Bernoulli,et al.  The most probable choice between several discrepant observations and the formation therefrom of the most likely induction , 1961 .

[4]  J. F. Gentleman,et al.  Detecting Outliers. II. Supplementing the Direct Analysis of Residuals , 1975 .

[5]  S. Sheather,et al.  Robust Nonparametric Methods , 2000 .

[6]  Peter J. Rousseeuw,et al.  The Remedian: A Robust Averaging Method for Large Data Sets , 1990 .

[7]  V. Yohai,et al.  A Fast Procedure for Outlier Diagnostics in Large Regression Problems , 1999 .

[8]  R. Randles,et al.  An Adaptive M-Estimator and Its Application to a Selection Problem , 1978 .

[9]  Irwin Guttman,et al.  Robustness to Spurious Observations of Linearized Hodges-Lehmann Estimators and Anscombe , 1980 .

[10]  An Historical Note on Recursive Residuals , 1978 .

[11]  Joseph W. McKean,et al.  Coefficients of determination for least absolute deviation analysis , 1987 .

[12]  Richard E. Lund,et al.  Tables for An Approximate Test for Outliers in Linear Models , 1975 .

[13]  D. G. Simpson,et al.  On One-Step GM Estimates and Stability of Inferences in Linear Regression , 1992 .

[14]  An outlier nomination method based on the multihalver , 2004 .

[15]  S. Chatterjee,et al.  Influential Observations, High Leverage Points, and Outliers in Linear Regression , 1986 .

[16]  H. Theil.  The Analysis of Disturbances in Regression Analysis , 1965 .

[17]  W. Fung.  Critical values for testing in multivariate statistical outliers , 1988 .

[18]  S. Chatterjee.  Sensitivity analysis in linear regression , 1988 .

[19]  H. A. David.  Contributions to Survey Sampling and Applied Statistics , 1978 .

[20]  N. Weiss.  A Course in Probability , 2005 .

[21]  Andrew H. Van de Ven,et al.  A community perspective on the emergence of innovations , 1993 .

[22]  Joshua D. Naranjo,et al.  Bounded influence rank regression , 1994 .

[23]  H. Theil.  A Rank-Invariant Method of Linear and Polynomial Regression Analysis , 1992 .

[24]  F. Scholz.  Weighted Median Regression Estimates , 1978 .

[25]  H. Theil.  Principles of econometrics , 1971 .

[26]  Douglas M. Hawkins.  Identification of Outliers , 1980, Monographs on Applied Probability and Statistics.

[27]  G. L. Sievers.  Weighted Rank Statistics for Simple Linear Regression , 1978 .

[28]  Y. Tse.  Assessing Lund's critical values for testing for outliers in linear regression models , 1988 .

[29]  D. F. Andrews,et al.  A Robust Method for Multiple Linear Regression , 1974 .

[30]  T. E. Dielman,et al.  A comparison of forecasts from least absolute value and least squares regression , 1986 .

[31]  William H. Swallow,et al.  Using Robust Scale Estimates in Detecting Multiple Outliers in Linear Regression , 1996 .

[32]  J. Durbin,et al.  Techniques for Testing the Constancy of Regression Relationships Over Time , 1975 .

[33]  J. L. Hodges,et al.  Estimates of Location Based on Rank Tests , 1963 .

[34]  V. Yohai,et al.  A class of robust and fully efficient regression estimators , 2002 .

[35]  M. Wilk,et al.  Detecting Outliers in a Two-Way Table: I. Statistical Behavior of Residuals* , 1975 .

[36]  M. M. Siddiqui,et al.  Robust Estimation of Location , 1967 .

[37]  Xizhi Wu,et al.  Robust Outlier Detection in Linear Regression , 2004 .

[38]  R. H. Stumpf,et al.  Graphical exploratory data analysis , 1986 .

[39]  Marianthi Markatou,et al.  A one-step robust estimator for regression based on the weighted likelihood reweighting scheme , 1998 .

[40]  W. Dixon,et al.  Estimates of Parameters of a Censored Regression Sample , 1972 .

[41]  F. Kianifard,et al.  Using Recursive Residuals, Calculated on Adaptively-Ordered Observations, to Identify Outliers in Linear Regression , 1989 .

[42]  Jim Freeman,et al.  Outliers in Statistical Data (3rd edition) , 1995 .

[43]  J. Tukey.  The Ninther, a Technique for Low-Effort Robust (Resistant) Location in Large Samples , 1978 .

[44]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[45]  Alan B. Forsythe,et al.  Robust Estimation of Straight Line Regression Coefficients by Minimizing pth Power Deviations , 1972 .

[46]  Benjamin Peirce,et al.  Criterion for the rejection of doubtful observations , 1852 .

[47]  Louis A. Jaeckel.  Estimating Regression Coefficients by Minimizing the Dispersion of the Residuals , 1972 .

[48]  John S. Ramberg,et al.  An Adaptive Multiple Regression Procedure Based on M-Estimators , 1980 .

[49]  E. J. Godolphin,et al.  Uncorrelated residuals and an exact test for two variance components in experimental design , 1992 .

[50]  Y. Dodge.  on Statistical data analysis based on the L1-norm and related methods , 1987 .

[51]  D. F. Andrews,et al.  Significance tests based on residuals , 1971 .

[52]  Victor J. Yohai,et al.  Functional stability of one-step GM-estimators in approximately linear regression , 1998 .

[53]  Irwin Guttman,et al.  Investigation of Rules for Dealing With Outliers in Small Samples from the Normal Distribution: I: Estimation of the Mean , 1969 .

[54]  P. Sprent,et al.  Non-Parametric Regression , 1983 .

[55]  William H. Swallow,et al.  A review of the development and application of recursive residuals in linear models , 1996 .

[56]  Mervyn G. Marasinghe.  A Multistage Procedure for Detecting Several Outliers in Linear Regression , 1985 .

[57]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[58]  A. Atkinson.  Fast Very Robust Methods for the Detection of Multiple Outliers , 1994 .

[59]  J. Wolfowitz,et al.  Introduction to the Theory of Statistics. , 1951 .

[60]  D. Montgomery,et al.  A comparative analysis of multiple outlier detection procedures in the linear regression model , 2001 .

[61]  S. Velilla.  A note on the behaviour of residual plots in regression , 1995 .

[62]  David E. Tyler,et al.  Constrained M-estimation for multivariate location and scatter , 1996 .

[63]  The Robustness of Efficiency of Adjusted Trimmed Estimators in Linear Regression , 1977 .

[64]  A Monte Carlo Study of Two Robust Alternatives of Least Square Regression Estimation , 1974 .

[65]  Pranab Kumar Sen,et al.  Invariance Principles for Recursive Residuals , 1982 .

[66]  Critical Values for Identifying Outliers , 1980 .

[67]  R. Doornbos.  Testing for a single outlier in a linear model , 1981 .

[68]  P. J. Huber.  Robust Estimation of a Location Parameter , 1964 .

[69]  R. H. Moore,et al.  Testing for a Single Outlier in Simple Linear Regression , 1973 .

[70]  J. Simonoff,et al.  Procedures for the Identification of Multiple Outliers in Linear Models , 1993 .

[71]  William S Agee,et al.  Application of Robust Regression to Trajectory Data Reduction , 1979 .

[72]  ADAPTIVE M-ESTIMATION PROCEDURE: ADEQUACY AND ALTERNATIVE APPROACH , 2002 .

[73]  Ali S. Hadi,et al.  [Influential Observations, High Leverage Points, and Outliers in Linear Regression]: Rejoinder , 1986 .

[74]  M. K. Gupta.  AN ASYMPTOTICALLY NONPARAMETRIC TEST OF SYMMETRY , 1967 .

[75]  Sanford Fidell,et al.  Effective masking bandwidths at low frequencies , 1979 .

[76]  Peter J. Huber,et al.  Between Robustness and Diagnostics , 1991 .

[77]  J. Ramsay.  A Comparative Study of Several Robust Estimates of Slope, Intercept, and Scale in Linear Regression , 1977 .