Kernel Conditional Moment Test via Maximum Moment Restriction

We propose a new family of specification tests called kernel conditional moment (KCM) tests. Our tests are built on a novel representation of conditional moment restrictions in a reproducing kernel Hilbert space (RKHS) called conditional moment embedding (CMME). After transforming the conditional moment restrictions into a continuum of unconditional counterparts, the test statistic is defined as the maximum moment restriction (MMR) within the unit ball of the RKHS. We show that the MMR not only fully characterizes the original conditional moment restrictions, leading to consistency in both hypothesis testing and parameter estimation, but also has an analytic expression that is easy to compute as well as closed-form asymptotic distributions. Our empirical studies show that the KCM test has a promising finite-sample performance compared to existing tests.

[1]  J. Sargan.  THE ESTIMATION OF ECONOMIC RELATIONSHIPS USING INSTRUMENTAL VARIABLES , 1958 .

[2]  Zhiwei Steven Wu,et al.  Orthogonal Random Forest for Causal Inference , 2018, ICML.

[3]  Ilias Zadik,et al.  Orthogonal Machine Learning: Power and Limitations , 2017, ICML.

[4]  Le Song,et al.  A Hilbert Space Embedding for Distributions , 2007, Discovery Science.

[5]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[6]  Whitney K. Newey,et al.  Maximum Likelihood Specification Testing and Conditional Moment Tests , 1985 .

[7]  J. Robins,et al.  Double/Debiased Machine Learning for Treatment and Structural Parameters , 2017 .

[8]  J. Muth.  Rational Expectations and the Theory of Price Movements , 1961 .

[9]  Neil D. Lawrence,et al.  Kernels for Vector-Valued Functions: a Review , 2011, Found. Trends Mach. Learn..

[10]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[11]  Yvik Swan,et al.  Stein’s density approach and information inequalities , 2012, 1210.3921.

[12]  W. Newey,et al.  Large sample estimation and hypothesis testing , 1986 .

[13]  Krikamol Muandet,et al.  Minimax Estimation of Kernel Mean Embeddings , 2016, J. Mach. Learn. Res..

[14]  Paul Janssen,et al.  Consistency of the Generalized Bootstrap for Degenerate $U$-Statistics , 1993 .

[15]  Suojin Wang,et al.  A simple consistent bootstrap test for a parametric regression function , 1998 .

[16]  Yuichi Kitamura,et al.  Testing conditional moment restrictions , 2003 .

[17]  Robert M. de Jong,et al.  THE BIERENS TEST UNDER DATA DEPENDENCE , 1996 .

[18]  Yuichi Kitamura,et al.  Empirical Likelihood Based Inference in Conditional Moment Restriction Models , 2004 .

[19]  Ignacio N. Lobato,et al.  Consistent Estimation of Models Defined by Conditional Moment Restrictions , 2004 .

[20]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[21]  Xiaohong Chen,et al.  Efficient Estimation of Models with Conditional Moment Restrictions Containing Unknown Functions , 2003 .

[22]  Stefan Wager,et al.  Estimation and Inference of Heterogeneous Treatment Effects using Random Forests , 2015, Journal of the American Statistical Association.

[23]  George Tauchen.  Diagnostic testing and evaluation of maximum likelihood models , 1985 .

[24]  H. White.  Consequences and Detection of Misspecified Nonlinear Regression Models , 1981 .

[25]  J. Florens,et al.  GENERALIZATION OF GMM TO A CONTINUUM OF MOMENT CONDITIONS , 2000, Econometric Theory.

[26]  C. Stein.  A bound for the error in the normal approximation to the distribution of a sum of dependent random variables , 1972 .

[27]  Arthur Gretton,et al.  Kernel Instrumental Variable Regression , 2019, NeurIPS.

[28]  Herman J. Bierens,et al.  A consistent conditional moment test of functional form , 1990 .

[29]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[30]  A Hausman Specification Test of Conditional Moment Restrictions , 2016 .

[31]  Bernhard Schölkopf,et al.  Kernel Mean Embedding of Distributions: A Review and Beyonds , 2016, Found. Trends Mach. Learn..

[32]  Ing Rj Ser.  Approximation Theorems of Mathematical Statistics , 1980 .

[33]  Vasilis Syrgkanis,et al.  Adversarial Generalized Method of Moments , 2018, ArXiv.

[34]  E. Kreyszig.  Introductory Functional Analysis With Applications , 1978 .

[35]  Qiang Liu,et al.  A Kernelized Stein Discrepancy for Goodness-of-fit Tests , 2016, ICML.

[36]  J. Escanciano.  A CONSISTENT DIAGNOSTIC TEST FOR REGRESSION MODELS USING PROJECTIONS , 2006, Econometric Theory.

[37]  W. Newey,et al.  16 Efficient estimation of models with conditional moment restrictions , 1993 .

[38]  Qi Li,et al.  Consistent Model Specification Tests : Kernel-Based Tests versus Bierens ' ICM Tests , 2008 .

[39]  I. Pinelis.  OPTIMUM BOUNDS FOR THE DISTRIBUTIONS OF MARTINGALES IN BANACH SPACES , 1994, 1208.2200.

[40]  Bernhard Schölkopf,et al.  Kernel Distribution Embeddings: Universal Kernels, Characteristic Kernels and Kernel Metrics on Distributions , 2016, J. Mach. Learn. Res..

[41]  Arthur Gretton,et al.  A Kernel Test of Goodness of Fit , 2016, ICML.

[42]  N. Chopin,et al.  Control functionals for Monte Carlo integration , 2014, 1410.2392.

[43]  L. Hansen.  LARGE SAMPLE PROPERTIES OF GENERALIZED METHOD OF , 1982 .

[44]  Andrew Bennett,et al.  Deep Generalized Method of Moments for Instrumental Variable Analysis , 2019, NeurIPS.

[45]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[46]  Guido W. Imbens,et al.  Empirical likelihood estimation and consistent tests with conditional moment restrictions , 2003 .

[47]  Krikamol Muandet,et al.  Dual IV: A Single Stage Instrumental Variable Regression , 2019, ArXiv.

[48]  Herman J. Bierens.  Econometric Model Specification: Consistent Model Specification Tests and Semi-Nonparametric Modeling and Inference , 2016 .

[49]  J. Zheng,et al.  A consistent test of functional form via nonparametric estimation techniques , 1996 .

[50]  E. Giné,et al.  On the Bootstrap of $U$ and $V$ Statistics , 1992 .

[51]  Whitney K. Newey,et al.  EFFICIENT INSTRUMENTAL VARIABLES ESTIMATION OF NONLINEAR MODELS , 1990 .

[52]  S. Athey,et al.  Generalized random forests , 2016, The Annals of Statistics.

[53]  Krikamol Muandet,et al.  Dual Instrumental Variable Regression , 2019, NeurIPS.

[54]  Khashayar Khosravi,et al.  Non-Parametric Inference Adaptive to Intrinsic Dimension , 2019, CLeaR.

[55]  Miguel A. Delgado,et al.  Consistent Tests of Conditional Moment Restrictions , 2006 .

[56]  Qi Li,et al.  CONSISTENT MODEL SPECIFICATION TESTS , 2000, Econometric Theory.

[57]  J. Hausman.  Specification tests in econometrics , 1978 .

[58]  Maxwell B. Stinchcombe,et al.  CONSISTENT SPECIFICATION TESTING WITH NUISANCE PARAMETERS PRESENT ONLY UNDER THE ALTERNATIVE , 1998, Econometric Theory.

[59]  H. Bierens.  Consistent model specification tests , 1982 .

[60]  A. Berlinet,et al.  Reproducing kernel Hilbert spaces in probability and statistics , 2004 .

[61]  S. Smale,et al.  Learning Theory Estimates via Integral Operators and Their Approximations , 2007 .

[62]  Winfried Stute,et al.  Nonparametric model checks for regression , 1997 .

[63]  Herman J. Bierens,et al.  Asymptotic Theory of Integrated Conditional Moment Tests , 1997 .

[64]  Christopher R. Taber,et al.  Generalized Method of Moments , 2020, Time Series Analysis.

[65]  Joshua D. Angrist,et al.  Mostly Harmless Econometrics: An Empiricist's Companion , 2008 .

[66]  Clayton D. Scott,et al.  Robust kernel density estimation , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[67]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[68]  Kevin Leyton-Brown,et al.  Deep IV: A Flexible Approach for Counterfactual Prediction , 2017, ICML.