Quasi-Bayesian analysis of nonparametric instrumental variables models

This paper aims at developing a quasi-Bayesian analysis of the nonparametric instrumental variables model, with a focus on the asymptotic properties of quasi-posterior distributions. In this paper, instead of assuming a distributional assumption on the data generating process, we consider a quasi-likelihood induced from the conditional moment restriction, and put priors on the function-valued parameter. We call the resulting posterior quasi-posterior, which corresponds to ``Gibbs posterior'' in the literature. Here we focus on priors constructed on slowly growing finite-dimensional sieves. We derive rates of contraction and a nonparametric Bernstein-von Mises type result for the quasi-posterior distribution, and rates of convergence for the quasi-Bayes estimator defined by the posterior expectation. We show that, with priors suitably chosen, the quasi-posterior distribution (the quasi-Bayes estimator) attains the minimax optimal rate of contraction (convergence, resp.). These results greatly sharpen the previous related work.

[1]  R. Kress.  Linear Integral Equations , 1989 .

[2]  J. Florens,et al.  Linear Inverse Problems in Structural Econometrics Estimation Based on Spectral Decomposition and Regularization , 2003 .

[3]  R. Bhatia.  Matrix Analysis , 1996 .

[4]  Wenxin Jiang,et al.  Posterior Consistency of Nonparametric Conditional Moment Restricted Models , 2010, 1105.4847.

[5]  Hanna K. Pikkarainen,et al.  Convergence Rates for Linear Inverse Problems in the Presence of an Additive Normal Noise , 2009 .

[6]  A. Tsybakov,et al.  Wavelets, approximation, and statistical applications , 1998 .

[7]  J. Florens,et al.  Regularized Posteriors in Linear Ill‐Posed Inverse Problems , 2012 .

[8]  Richard Nickl,et al.  Rates of contraction for posterior distributions in Lr-metrics, 1 ≤ r ≤ ∞ , 2011, 1203.2043.

[9]  L. Cavalier.  Nonparametric statistical inverse problems , 2008 .

[10]  A. V. D. Vaart,et al.  Misspecification in infinite-dimensional Bayesian statistics , 2006, math/0607023.

[11]  W. Newey,et al.  Instrumental variable estimation of nonparametric models , 2003 .

[12]  Catia Scricciolo.  Convergence rates for Bayesian density estimation of infinite-dimensional exponential families , 2006, 0708.0175.

[13]  P. Massart,et al.  About the constants in Talagrand's concentration inequalities for empirical processes , 2000 .

[14]  A. W. Vaart,et al.  Bayes procedures for adaptive inference in inverse problems for the white noise model , 2012, Probability Theory and Related Fields.

[15]  Susanne M. Schennach,et al.  Bayesian exponentially tilted empirical likelihood , 2005 .

[16]  A. Belloni,et al.  On the Computational Complexity of MCMC-Based Estimators in Large Samples , 2007, 0704.2167.

[17]  Xiaohong Chen,et al.  Estimation of Nonparametric Conditional Moment Models with Possibly Nonsmooth Generalized Residuals , 2009 .

[18]  J. Doob.  Stochastic processes , 1953 .

[19]  Jae-Young Kim,et al.  Limited information likelihood and Bayesian analysis , 2002 .

[20]  Stig Larsson,et al.  Posterior Contraction Rates for the Bayesian Approach to Linear Ill-Posed Inverse Problems , 2012, 1203.5753.

[21]  Joel L. Horowitz,et al.  Nonparametric Instrumental Variables Estimation of a Quantile Regression Model , 2006 .

[22]  Subhashis Ghosal,et al.  Asymptotic normality of posterior distributions in high-dimensional linear models , 1999 .

[23]  V. Chernozhukov,et al.  Nonparametric Instrumental Variable Estimators of Structural Quantile Effects , 2011 .

[24]  Xavier D'Haultfoeuille,et al.  ON THE COMPLETENESS CONDITION IN NONPARAMETRIC INSTRUMENTAL PROBLEMS , 2010, Econometric Theory.

[25]  M. Rudelson.  Random Vectors in the Isotropic Position , 1996, math/9608208.

[26]  J. Horowitz.  Applied Nonparametric Instrumental Variables Estimation , 2011 .

[27]  B. Clarke,et al.  Reference priors for exponential families with increasing dimension , 2010 .

[28]  A. V. D. Vaart,et al.  Convergence rates of posterior distributions for non-i.i.d. observations , 2007, 0708.0491.

[29]  P. Gänssler.  Weak Convergence and Empirical Processes - A. W. van der Vaart; J. A. Wellner. , 1997 .

[30]  A. W. van der Vaart,et al.  Bayesian Recovery of the Initial Condition for the Heat Equation , 2011, 1111.5876.

[31]  Dominique Bontemps,et al.  Bernstein von Mises Theorems for Gaussian Regression with increasing number of regressors , 2010, 1009.1370.

[32]  D. Cox.  An Analysis of Bayesian Inference for Nonparametric Regression , 1993 .

[33]  V. Chernozhukov,et al.  An MCMC Approach to Classical Estimation , 2002, 2301.07782.

[34]  Andreas Hofinger,et al.  Convergence rate for the Bayesian approach to linear inverse problems , 2007 .

[35]  Tong Zhang,et al.  Information-theoretic upper and lower bounds for statistical estimation , 2006, IEEE Transactions on Information Theory.

[36]  A. V. D. Vaart,et al.  BAYESIAN INVERSE PROBLEMS WITH GAUSSIAN PRIORS , 2011, 1103.2692.

[37]  A. V. D. Vaart,et al.  Asymptotic Statistics: Frontmatter , 1998 .

[38]  Andres Santos,et al.  Inference in Nonparametric Instrumental Variables With Partial Identification , 2012 .

[39]  L. Wasserman,et al.  Rates of convergence of posterior distributions , 2001 .

[40]  A. V. D. Vaart,et al.  Convergence rates of posterior distributions , 2000 .

[41]  M. Tanner,et al.  Gibbs posterior for variable selection in high-dimensional classification and data mining , 2008, 0810.5655.

[42]  J. Florens,et al.  Nonparametric Instrumental Regression , 2010 .

[43]  Yingyao Hu,et al.  NONPARAMETRIC IDENTIFICATION USING INSTRUMENTAL VARIABLES: SUFFICIENT CONDITIONS FOR COMPLETENESS , 2011, Econometric Theory.

[44]  Jun S. Liu,et al.  Implementation of Estimating Function-Based Inference Procedures With Markov Chain Monte Carlo Samplers , 2007 .

[45]  Xiaohong Chen,et al.  ON RATE OPTIMALITY FOR ILL-POSED INVERSE PROBLEMS IN ECONOMETRICS , 2007, Econometric Theory.

[46]  Andrew M. Stuart,et al.  Inverse problems: A Bayesian perspective , 2010, Acta Numerica.

[47]  Victor Chernozhukov,et al.  Posterior Inference in Curved Exponential Families Under Increasing Dimensions , 2007, 0904.3132.

[48]  Xiaohong Chen,et al.  Semi‐Nonparametric IV Estimation of Shape‐Invariant Engel Curves , 2003 .

[49]  Specification testing in nonparametric instrumental variable estimation , 2012 .

[50]  M. Talagrand.  New concentration inequalities in product spaces , 1996 .

[51]  Tong Zhang.  From ɛ-entropy to KL-entropy: Analysis of minimum information complexity density estimation , 2006, math/0702653.

[52]  S. Ghosal.  Asymptotic Normality of Posterior Distributions for Exponential Families when the Number of Parameters Tends to Infinity , 2000 .

[53]  T. Severini,et al.  SOME IDENTIFICATION ISSUES IN NONPARAMETRIC LINEAR MODELS WITH ENDOGENOUS REGRESSORS , 2006, Econometric Theory.

[54]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[55]  Xiaohong Chen,et al.  Efficient Estimation of Models with Conditional Moment Restrictions Containing Unknown Functions , 2003 .

[56]  Demian Pouzo,et al.  Estimation of Nonparametric Conditional Moment Models with Possibly Nonsmooth Moments , 2008 .

[57]  Identification via completeness for discrete covariates and orthogonal polynomials , 2012 .

[58]  P. Hall,et al.  Nonparametric methods for inference in the presence of instrumental variables , 2003, math/0603130.

[59]  J. Craggs.  Applied Mathematical Sciences , 1973 .

[60]  S. Boucheron,et al.  A Bernstein-Von Mises Theorem for discrete probability distributions , 2008, 0807.2096.

[61]  I. Daubechies,et al.  Wavelets on the Interval and Fast Wavelet Transforms , 1993 .

[62]  Anna Simoni,et al.  Nonparametric Estimation of An Instrumental Regression: A Quasi-Bayesian Approach Based on Regularized Posterior , 2012 .