Semiparametrically weighted robust estimation of regression models

A class of two-step robust regression estimators that achieve a high relative efficiency for data from light-tailed, heavy-tailed, and contaminated distributions irrespective of the sample size is proposed and studied. In particular, the least weighted squares (LWS) estimator is combined with data-adaptive weights, which are determined from the empirical distribution or quantile functions of regression residuals obtained from an initial robust fit. Just like many existing two-step robust methods, the LWS estimator with the proposed weights preserves robust properties of the initial robust estimate. However, contrary to the existing methods and despite the data-dependent weights, the first-order asymptotic behavior of LWS is fully independent of the initial estimate under mild conditions. Moreover, the proposed estimation method is asymptotically efficient if errors are normally distributed. A simulation study documents these theoretical properties in finite samples; in particular, the relative efficiency of LWS with the proposed weighting schemes can reach 85%-100% in samples of several tens of observations under various distributional models.

[1]  L. Fernholz.  Reducing the variance by smoothing , 1997 .

[2]  P. Rousseeuw.  Multivariate estimation with high breakdown point , 1985 .

[3]  Georg Ch. Pflug,et al.  Mathematical statistics and applications , 1985 .

[4]  Ola Hössjer,et al.  On the optimality of S-estimators☆ , 1992 .

[5]  P. L. Davies,et al.  Breakdown and groups , 2005, math/0508497.

[6]  Peter J. Rousseeuw,et al.  ROBUST REGRESSION BY MEANS OF S-ESTIMATORS , 1984 .

[7]  Andre Lucas,et al.  Testing for ARCH in the presence of additive outliers , 1999 .

[8]  D. G. Simpson,et al.  On One-Step GM Estimates and Stability of Inferences in Linear Regression , 1992 .

[9]  P. Rousseeuw.  Least Median of Squares Regression , 1984 .

[10]  PETER J. ROUSSEEUW,et al.  Computing LTS Regression for Large Data Sets , 2005, Data Mining and Knowledge Discovery.

[11]  Kang-Mo Jung.  Multivariate least-trimmed squares regression estimator , 2005, Comput. Stat. Data Anal..

[12]  E. Ronchetti,et al.  A journey in single steps: robust one-step M-estimation in linear regression , 2002 .

[13]  Arie Preminger,et al.  Forecasting Exchange Rates: A Robust Regression Approach , 2005 .

[14]  Werner A. Stahel,et al.  Robust Statistics: The Approach Based on Influence Functions , 1987 .

[15]  Jonathan R.W. Temple,et al.  Robustness Tests of the Augmented Solow Model , 1998 .

[16]  V. Yohai.  HIGH BREAKDOWN-POINT AND HIGH EFFICIENCY ROBUST ESTIMATES FOR REGRESSION , 1987 .

[17]  E. Ronchetti,et al.  Robust inference with GMM estimators , 2001 .

[18]  Pavel Čížek,et al.  Reweighted least trimmed squares: an alternative to one-step estimators , 2010, TEST.

[19]  V. Yohai,et al.  Robust regression with both continuous and categorical predictors , 2000 .

[20]  V. Yohai,et al.  A class of robust and fully efficient regression estimators , 2002 .

[21]  Pavel Čížek,et al.  Least trimmed squares in nonlinear regression under dependence , 2006 .

[22]  Jan Ámos Víšek.  The Least Weighted Squares I. The Asymptotic Linearity Of Normal Equations , 2002 .

[23]  A. Mokkadem.  Mixing properties of ARMA processes , 1988 .

[24]  J. Davidson.  Stochastic Limit Theory , 1994 .

[25]  T. Fomby,et al.  Large Shocks, Small Shocks, and Economic Fluctuations: Outliers in Macroeconomic Time Series , 1994 .

[26]  C. R. Rao,et al.  Handbook of Statistics 15: Robust Inference , 2000, Technometrics.

[27]  P. Čížek.  GENERAL TRIMMED ESTIMATION: ROBUST APPROACH TO NONLINEAR AND LIMITED DEPENDENT VARIABLE MODELS , 2008, Econometric Theory.

[28]  P. Rousseeuw.  5 Introduction to positive-breakdown methods , 1997 .

[29]  Marc G. Genton,et al.  Comprehensive definitions of breakdown points for independent and dependent observations , 2003 .

[30]  Mia Hubert,et al.  Robust regression with both continuous and binary regressors , 1997 .

[31]  V. Yohai,et al.  High Breakdown-Point Estimates of Regression by Means of the Minimization of an Efficient Scale , 1988 .

[32]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[33]  Robin Nunkesser,et al.  An evolutionary algorithm for robust regression , 2010, Comput. Stat. Data Anal..

[34]  P. L. Davies,et al.  The asymptotics of S-estimators in the linear regression model , 1990 .

[35]  Stephen Portnoy,et al.  Reweighted LS Estimators Converge at the same Rate as the Initial Estimator , 1992 .

[36]  Jaejoon Woo.  Economic, Political and Institutional Determinants of Public Deficits , 2001 .

[37]  Stefan Van Aelst,et al.  Fast and robust bootstrap for LTS , 2005, Comput. Stat. Data Anal..

[38]  P. Čížek,et al.  Efficient Robust Estimation of Regression Models , 2007 .

[39]  B. Nielsen,et al.  The Empirical Process of Autoregressive Residuals , 2007 .

[40]  Jan Ámos Víšek.  The Least Weighted Squares Ii. Consistency And Asymptotic Normality , 2002 .

[41]  A. Ullah,et al.  Nonparametric Econometrics , 1999 .

[42]  D. Andrews.  Laws of Large Numbers for Dependent Non-Identically Distributed Random Variables , 1988, Econometric Theory.

[43]  A. Stromberg,et al.  The Least Trimmed Differences Regression Estimator and Alternatives , 2000 .

[44]  Shinichi Sakata,et al.  HIGH BREAKDOWN POINT CONDITIONAL DISPERSION ESTIMATION WITH APPLICATION TO S&P 500 DAILY RETURNS VOLATILITY , 1998 .

[45]  E. Ronchetti,et al.  Robust Estimators for Simultaneous Equations Models , 1997 .