Regression Model Estimation Using Least Absolute Deviations , Least Squares Deviations and Minimax Absolute Deviations Criteria

Regression models and their statistical analyses is the most important tool used by scientists in data analyses especially for modeling the relationship among random variables and making predictions with higher accuracy. A fundamental problem in the theory of errors, which has drawn attention of leading mathematicians and scientists since past few centuries, was that of fitting functions. For the pioneering work to develop procedures for fitting functions, in the eighteenth century, we refer to the research by Mayer, Boscovich, Laplace, Legendre, Simpson, Gauss, and Euler. They worked on the methods of least absolute deviations, least squares deviations and minimax absolute deviations. Today’s widely celebrated procedure of the method of least squares for function fitting is credited to the published works of Legendre and Gauss [1][2]. The least square estimates are best linear unbiased estimates and are optimal under assumptions that the errors follow normal distributions, are free of large size outliers and satisfy the GaussMarkov assumptions. However, the least squares based models in practice may fail to provide optimal results in non-Gaussian situations especially when the errors follow distributions with the fat tails. In this paper, we will present an overview of some important works in fitting linear relationship. We will also statistical properties of the Lp -norm based model estimation, measures of the model adequacy and also future research questions of interest. Keywords— Regression model, Least squares estimates, Least absolute deviations, Minimax absolute deviations, Lp-norm. I. FITTING FUNCTIONS: INTRODUCTION AND REVIEW standard linear regression model expressing the linear relationship between the study variable and explanatory variables is defined by linear equations:

[1]  Peter C. B. Phillips,et al.  A Shortcut to LAD Estimator Asymptotics , 1991, Econometric Theory.

[2]  Adrien-Marie Legendre,et al.  Nouvelles méthodes pour la détermination des orbites des comètes , 1970 .

[3]  Subhash C. NarulaI,et al.  The Minimum Sum of Absolute Errors Regression: A State of the Art Survey , 1982 .

[4]  Andrew A. Weiss,et al.  Estimating Nonlinear Dynamic Models Using Least Absolute Error Estimation , 1991, Econometric Theory.

[5]  D. Pollard.  Asymptotics for Least Absolute Deviation Regression Estimators , 1991, Econometric Theory.

[6]  Nabih N. Abdelmalek,et al.  Linear L1 Approximation for a Discrete Point Set and L1 Solutions of Overdetermined Linear Equations , 1971, J. ACM.

[7]  T. D. Wallace,et al.  A Sampling Study of Minimum Absolute Deviations Estimators , 1963 .

[8]  Pranesh Kumar,et al.  On the Lp-Norm Regression Models for Estimating Value-at-Risk , 2015, Serdica Journal of Computing.

[9]  Ray C. Fair.  8 Estimating and Testing the US Model , 1994 .

[10]  Carolo Friederico Gauss.  Theoria Motus Corporum Coelestium in Sectionibus Conicis Solem Ambientium , 2014 .

[11]  Richard A. Davis,et al.  Least absolute deviation estimation for all-pass time series models , 2001 .

[12]  Robert Bartels,et al.  On the Use of Limit Theorem Arguments in Economic Statistics , 1977 .

[13]  D. F. Andrews,et al.  A Robust Method for Multiple Linear Regression , 1974 .

[14]  Instrumental Variable Estimation Based on Mean Absolute Deviation , 1998 .

[15]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[16]  Yuehua Wu,et al.  On necessary conditions for the weak consistency of minimum L1-norm estimates in linear models , 1997 .

[17]  Marilena Furno.  LAD estimation with random coefficient autocorrelated errors , 2001 .

[18]  R. Koenker,et al.  Asymptotic Theory of Least Absolute Error Regression , 1978 .

[19]  J. D. Khazzoom,et al.  An Indirect Least Squares Estimator for Overidentified Equations , 1976 .

[20]  Takeshi Amemiya,et al.  Two Stage Least Absolute Deviations Estimators , 1982 .

[21]  H. G. Wilson,et al.  LEAST SQUARES VERSUS MINIMUM ABSOLUTE DEVIATIONS ESTIMATION IN LINEAR MODELS , 1978 .

[22]  J. G. Hunt,et al.  The Small Sample Properties of Simultaneous Equation Least Absolute Estimators vis-a-vis Least Squares Estimators , 1970 .

[23]  Hans Nyquist.  Recent Studies on Lp-Norm Estimation , 1980 .

[24]  J. Powell,et al.  Least absolute deviations estimation for the censored regression model , 1984 .

[25]  P. Laplace.  Théorie analytique des probabilités , 1995 .

[26]  Madhuchhanda Dasgupta,et al.  Least Absolute Deviation Estimation of Linear Econometric Models: A Literature Review , 2004 .