Semiparametrically weighted robust estimation of regression models

A class of two-step robust regression estimators that achieve a high relative efficiency for data from light-tailed, heavy-tailed, and contaminated distributions irrespective of the sample size is proposed and studied. In particular, the least weighted squares (LWS) estimator is combined with data-adaptive weights, which are determined from the empirical distribution or quantile functions of regression residuals obtained from an initial robust fit. Just like many existing two-step robust methods, the LWS estimator with the proposed weights preserves robust properties of the initial robust estimate. However, contrary to the existing methods and despite the data-dependent weights, the first-order asymptotic behavior of LWS is fully independent of the initial estimate under mild conditions. Moreover, the proposed estimationmethod is asymptotically efficient if errors are normally distributed. A simulation study documents these theoretical properties in finite samples; in particular, the relative efficiency of LWS with the proposed weighting schemes can reach 85%–100% in samples of several tens of observations under various distributional models. © 2010 Elsevier B.V. All rights reserved.

[1]  P. Čížek.  GENERAL TRIMMED ESTIMATION: ROBUST APPROACH TO NONLINEAR AND LIMITED DEPENDENT VARIABLE MODELS , 2007, Econometric Theory.

[2]  T. Fomby,et al.  Large Shocks, Small Shocks, and Economic Fluctuations: Outliers in Macroeconomic Time Series , 1994 .

[3]  B. Nielsen,et al.  The Empirical Process of Autoregressive Residuals , 2007 .

[4]  L. Fernholz.  Reducing the variance by smoothing , 1997 .

[5]  Stefan Van Aelst,et al.  Fast and robust bootstrap for LTS , 2005, Comput. Stat. Data Anal..

[6]  E. Ronchetti,et al.  A journey in single steps: robust one-step M-estimation in linear regression , 2002 .

[7]  P. Rousseeuw.  Least Median of Squares Regression , 1984 .

[8]  Ola Hössjer,et al.  On the optimality of S-estimators☆ , 1992 .

[9]  Marc G. Genton,et al.  Comprehensive definitions of breakdown points for independent and dependent observations , 2003 .

[10]  A. Stromberg,et al.  The Least Trimmed Differences Regression Estimator and Alternatives , 2000 .

[11]  PETER J. ROUSSEEUW,et al.  Computing LTS Regression for Large Data Sets , 2005, Data Mining and Knowledge Discovery.

[12]  Shinichi Sakata,et al.  HIGH BREAKDOWN POINT CONDITIONAL DISPERSION ESTIMATION WITH APPLICATION TO S&P 500 DAILY RETURNS VOLATILITY , 1998 .

[13]  P. L. Davies,et al.  Breakdown and groups , 2005, math/0508497.

[14]  E. Ronchetti,et al.  Robust inference with GMM estimators , 2001 .

[15]  P. Čížek,et al.  Efficient Robust Estimation of Regression Models , 2007 .

[16]  E. Ronchetti,et al.  Robust Estimators for Simultaneous Equations Models , 1997 .

[17]  V. Yohai.  HIGH BREAKDOWN-POINT AND HIGH EFFICIENCY ROBUST ESTIMATES FOR REGRESSION , 1987 .

[18]  V. Yohai,et al.  A class of robust and fully efficient regression estimators , 2002 .

[19]  Pavel Čížek,et al.  Least trimmed squares in nonlinear regression under dependence , 2006 .

[20]  A. Mokkadem.  Mixing properties of ARMA processes , 1988 .

[21]  Robin Nunkesser,et al.  An evolutionary algorithm for robust regression , 2010, Comput. Stat. Data Anal..

[22]  Jan Ámos Víšek.  The Least Weighted Squares I. The Asymptotic Linearity Of Normal Equations , 2002 .

[23]  Stephen Portnoy,et al.  Reweighted LS Estimators Converge at the same Rate as the Initial Estimator , 1992 .

[24]  Jaejoon Woo.  Economic, Political and Institutional Determinants of Public Deficits , 2001 .

[25]  P. L. Davies,et al.  The asymptotics of S-estimators in the linear regression model , 1990 .

[26]  Andre Lucas,et al.  Testing for ARCH in the presence of additive outliers , 1999 .

[27]  Mia Hubert,et al.  Robust regression with both continuous and binary regressors , 1997 .

[28]  Jonathan R.W. Temple,et al.  Robustness Tests of the Augmented Solow Model , 1998 .

[29]  V. Yohai,et al.  High Breakdown-Point Estimates of Regression by Means of the Minimization of an Efficient Scale , 1988 .

[30]  Kang-Mo Jung.  Multivariate least-trimmed squares regression estimator , 2005, Comput. Stat. Data Anal..

[31]  J. Davidson.  Stochastic Limit Theory , 1994 .

[32]  Jan Ámos Víšek.  The Least Weighted Squares Ii. Consistency And Asymptotic Normality , 2002 .

[33]  Arie Preminger,et al.  Forecasting Exchange Rates: A Robust Regression Approach , 2005 .

[34]  V. Yohai,et al.  Robust regression with both continuous and categorical predictors , 2000 .

[35]  D. G. Simpson,et al.  On One-Step GM Estimates and Stability of Inferences in Linear Regression , 1992 .

[36]  Aman Ullah,et al.  Nonparametric Econometrics: Introduction , 1999 .

[37]  D. Andrews.  Laws of Large Numbers for Dependent Non-Identically Distributed Random Variables , 1988, Econometric Theory.

[38]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.