A regularization approach to the many instruments problem

This paper focuses on the estimation of a finite dimensional parameter in a linear model where the number of instruments is very large or infinite. In order to improve the small sample properties of standard instrumental variable (IV) estimators, we propose three modified IV estimators based on three different ways of inverting the covariance matrix of the instruments. These inverses involve a regularization or smoothing parameter. It should be stressed that no restriction on the number of instruments is needed and that all the instruments are used in the estimation. We show that the three estimators are asymptotically normal and attain the semiparametric efficiency bound. Higher-order analysis of the MSE reveals that the bias of the modified estimators does not depend on the number of instruments. Finally, we suggest a data-driven method for selecting the regularization parameter. Interestingly, our regularization techniques lead to a consistent nonparametric estimation of the optimal instrument.

[1]  J. Stock,et al.  Macroeconomic Forecasting Using Diffusion Indexes , 2002 .

[2]  Oliver Linton,et al.  An Alternative Way of ComputingEfficient Instrumental VariableEstimators , 2009 .

[3]  Takeshi Amemiya,et al.  ON THE USE OF PRINCIPAL COMPONENTS OF INDEPENDENT VARIABLES IN TWO-STAGE LEAST-SQUARES ESTIMATION* , 1966 .

[4]  Gary Chamberlain,et al.  FUNDS, FACTORS, AND DIVERSIFICATION IN ARBITRAGE PRICING MODELS , 1983 .

[5]  Chew Tuan Seng.  On Nonlinear Integrals , 1988 .

[6]  G. Wahba.  Improper Priors, Spline Smoothing and the Problem of Guarding Against Model Errors in Regression , 1978 .

[7]  G. Wahba.  Smoothing noisy data with spline functions , 1975 .

[8]  M. Stone.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[9]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[10]  A. Berlinet,et al.  Reproducing kernel Hilbert spaces in probability and statistics , 2004 .

[11]  C. Mallows.  Some Comments on Cp , 2000, Technometrics.

[12]  W. Newey,et al.  16 Efficient estimation of models with conditional moment restrictions , 1993 .

[13]  Stephen G. Donald,et al.  Choosing the Number of Instruments , 2001 .

[14]  Optimal Instrumental Variables Estimation For ARMA Models , 2001 .

[15]  Ignacio N. Lobato,et al.  Consistent Estimation of Models Defined by Conditional Moment Restrictions , 2004 .

[16]  J. Angrist,et al.  Does Compulsory School Attendance Affect Schooling and Earnings? , 1990 .

[17]  M. C. Jones,et al.  Spline Smoothing and Nonparametric Regression. , 1989 .

[18]  P. Chaussé.  Generalized empirical likelihood for a continuum of moment conditions , 2011 .

[19]  Ryo Okui,et al.  Instrumental variable estimation in the presence of many moment conditions , 2011 .

[20]  C. Mallows.  More comments on C p , 1995 .

[21]  P. Schmidt,et al.  GMM estimators with improved finite sample properties using principal components of the weighting matrix, with an application to the dynamic panel data model , 2006 .

[22]  Whitney K. Newey,et al.  Higher Order Properties of Gmm and Generalized Empirical Likelihood Estimators , 2003 .

[23]  Peter Craven,et al.  Smoothing noisy data with spline functions , 1978 .

[24]  Ker-Chau Li,et al.  Asymptotic Optimality for $C_p, C_L$, Cross-Validation and Generalized Cross-Validation: Discrete Index Set , 1987 .

[25]  O. Linton.  Edgeworth Approximations for Semiparametric Instrumental Variable Estimators and Test Statistics , 2000 .

[26]  J. Bai,et al.  Determining the Number of Factors in Approximate Factor Models , 2000 .

[27]  H. Theil,et al.  Economic Forecasts and Policy. , 1959 .

[28]  A. L. Nagar.  The Bias and Moment Matrix of the General k-Class Estimators of the Parameters in Simultaneous Equations , 1959 .

[29]  J. Florens,et al.  Efficient estimation of general dynamic models with a continuum of moment conditions , 2007 .

[30]  Paul A. Bekker,et al.  ALTERNATIVE APPROXIMATIONS TO THE DISTRIBUTIONS OF INSTRUMENTAL VARIABLE ESTIMATORS , 1994 .

[31]  Jean-Pierre Florens,et al.  ON THE ASYMPTOTIC EFFICIENCY OF GMM , 2013, Econometric Theory.

[32]  G. Kuersteiner.  Kernel-weighted GMM estimators for linear time series models , 2012 .

[33]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[34]  Ker-Chau Li,et al.  Asymptotic optimality of CL and generalized cross-validation in ridge regression with application to spline smoothing , 1986 .

[35]  M. Dagenais,et al.  Higher moment estimators for linear regression models with errors in the variables , 1997 .

[36]  J. Florens,et al.  GENERALIZATION OF GMM TO A CONTINUUM OF MOMENT CONDITIONS , 2000, Econometric Theory.

[37]  A. Owen.  Empirical likelihood ratio confidence intervals for a single functional , 1988 .

[38]  Brian J. Eastwood,et al.  Adaptive Rules for Seminonparametric Estimators That Achieve Asymptotic Normality , 1991, Econometric Theory.

[39]  C. L. Mallows.  Some comments on C_p , 1973 .

[40]  Miguel A. Delgado,et al.  Consistent Tests of Conditional Moment Restrictions , 2006 .

[41]  MEAN SQUARED ERROR REDUCTION FOR GMM ESTIMATORS OF LINEAR TIME SERIES MODELS , 2002 .

[42]  Christian Hansen,et al.  Estimation With Many Instrumental Variables , 2006, Journal of Business & Economic Statistics.

[43]  G. Kuersteiner.  Optimal Instrumental Variables Estimation for Arma Models , 1999 .

[44]  Bent E. Sørensen,et al.  GMM Estimation of a Stochastic Volatility Model: A Monte Carlo Study , 1996 .

[45]  R. Kanwal.  Linear Integral Equations , 1925, Nature.

[46]  Serena Ng,et al.  INSTRUMENTAL VARIABLE ESTIMATION IN A DATA RICH ENVIRONMENT , 2010, Econometric Theory.

[47]  Gene H. Golub,et al.  Generalized cross-validation as a method for choosing a good ridge parameter , 1979, Milestones in Matrix Computation.

[48]  M. Arellano,et al.  Some Tests of Specification for Panel Data: Monte Carlo Evidence and an Application to Employment Equations , 1991 .

[49]  B. Yandell.  Spline smoothing and nonparametric regression , 1989 .

[50]  M. Rothschild,et al.  Arbitrage, Factor Structure, and Mean-Variance Analysis on Large Asset Markets , 1982 .

[51]  Yuichi Kitamura,et al.  Empirical Likelihood Based Inference in Conditional Moment Restriction Models , 2004 .

[52]  Guido W. Imbens,et al.  Empirical likelihood estimation and consistent tests with conditional moment restrictions , 2003 .

[53]  J. Florens,et al.  Linear Inverse Problems in Structural Econometrics Estimation Based on Spectral Decomposition and Regularization , 2003 .

[54]  Oliver Linton,et al.  An Alternative Way of Computing Efficient Instrumental Variable Estimators , 2009 .