Gaussian Processes and Bayesian Moment Estimation

Given a set of moment restrictions that characterize a parameter ?, we investigate a semiparametric Bayesian approach for estimation of ? that imposes these moment restrictions in the nonparametric prior for the data distribution. As main contribution, we construct a degenerate Gaussian process prior for the density function associated with the data distribution F that imposes overidentifying restrictions. We show that this prior is computationally convenient. Since the likelihood function is not speci?ed by the model we construct it based on a linear functional transformation of F that has an asymptotically Gaussian empirical counterpart. This likelihood is used to construct the posterior distribution. We provide a frequentist validation of our procedure by showing: consistency of the maximum a posteriori estimator for ?, consistency and asymptotic normality of the posterior distribution of ?.

[1]  Wenxin Jiang,et al.  Posterior Consistency of Nonparametric Conditional Moment Restricted Models , 2010, 1105.4847.

[2]  Jeffrey M. Wooldridge,et al.  Introductory Econometrics: A Modern Approach , 1999 .

[3]  Ingrid Van Keilegom,et al.  A review on empirical likelihood methods for regression , 2009 .

[4]  H. Kuo.  Gaussian Measures in Banach Spaces , 1975 .

[5]  Anna Simoni,et al.  Online appendix to : Bayesian Estimation and Comparison of Moment Condition Models , 2017 .

[6]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[7]  L. Hansen,et al.  Finite Sample Properties of Some Alternative Gmm Estimators , 2015 .

[8]  L. Hansen.  Large Sample Properties of Generalized Method of Moments Estimators , 1982 .

[9]  E. Regazzini,et al.  Distribution Functions of Means of a Dirichlet Process , 1990 .

[10]  Umberto Amato,et al.  Maximum entropy regularization of Fredholm integral equations of the first kind , 1991 .

[11]  Yuichi Kitamura,et al.  Empirical likelihood methods with weakly dependent processes , 1997 .

[12]  J. Rousseau,et al.  BERNSTEIN-VON MISES THEOREM FOR LINEAR FUNCTIONALS OF THE DENSITY , 2009, 0908.4167.

[13]  J. Florens,et al.  Regularized Posteriors in Linear Ill‐Posed Inverse Problems , 2012 .

[14]  On the Distribution of Jumps of the Dirichlet Process , 1992 .

[15]  Susanne M. Schennach,et al.  Accompanying document to "Point Estimation with Exponentially Tilted Empirical Likelihood" , 2005, math/0512181.

[16]  G. Imbens,et al.  Information Theoretic Approaches to Inference in Moment Condition Models , 1995 .

[17]  G. Imbens,et al.  Nonparametric Applications of Bayesian Inference , 1996 .

[18]  Michel Broniatowski,et al.  Divergences and Duality for Estimation and Test under Moment Condition Models , 2010, 1002.0730.

[19]  George Judge,et al.  Asymptotic equivalence of empirical likelihood and Bayesian MAP , 2009, 0908.3397.

[20]  Susanne M. Schennach,et al.  Bayesian exponentially tilted empirical likelihood , 2005 .

[21]  D. Mondal,et al.  Hamiltonian Monte Carlo sampling in Bayesian empirical likelihood computation , 2017 .

[22]  Yuichi Kitamura,et al.  An Information-Theoretic Alternative to Generalized Method of Moments Estimation , 1997 .

[23]  J. N. K. Rao,et al.  Bayesian pseudo-empirical-likelihood intervals for complex surveys , 2010 .

[24]  Giuseppe Ragusa,et al.  Bayesian Likelihoods for Moment Condition Models , 2007 .

[25]  P. Bickel,et al.  The semiparametric Bernstein-von Mises theorem , 2010, 1007.0179.

[26]  I. Castillo.  A semiparametric Bernstein–von Mises theorem for Gaussian process priors , 2012 .

[27]  Jean-Pierre Florens,et al.  Elements of Bayesian Statistics , 1990 .

[28]  Eric Zivot,et al.  Bayesian and Classical Approaches to Instrumental Variables Regression , 2003 .

[29]  Minchul Shin,et al.  Bayesian Empirical Likelihood Estimation and Comparison of Moment Condition Models , 2016 .

[30]  Guido W. Imbens,et al.  One-step estimators for over-identified generalized method of moments models , 1997 .

[31]  Magoichirô Watanabe.  On a Linear Integral Equation , 1915 .

[32]  N. Lazar.  Bayesian empirical likelihood , 2003 .

[33]  Luke Bornn,et al.  Moment conditions and Bayesian non‐parametrics , 2015, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[34]  E. L. Lehmann,et al.  Theory of point estimation , 1950 .

[35]  Jean-Marie Rolin.  Some useful properties of the Dirichlet process , 1992 .

[36]  Jean-Pierre Florens,et al.  ON THE ASYMPTOTIC EFFICIENCY OF GMM , 2013, Econometric Theory.

[37]  Alexander Shapiro,et al.  Optimization Problems with Perturbations: A Guided Tour , 1998, SIAM Rev..

[38]  J. Lawless,et al.  Empirical Likelihood and General Estimating Equations , 1994 .

[39]  I. Csiszár.  $I$-Divergence Geometry of Probability Distributions and Minimization Problems , 1975 .

[40]  V. Chernozhukov,et al.  An MCMC Approach to Classical Estimation , 2002, 2301.07782.

[41]  A. Ronald Gallant,et al.  Bayesian estimation of state space models using moment conditions , 2017 .

[42]  A. Owen.  Empirical likelihood ratio confidence intervals for a single functional , 1988 .

[43]  Anna Simoni,et al.  REGULARIZING PRIORS FOR LINEAR INVERSE PROBLEMS , 2014, Econometric Theory.

[44]  Chan‐Fu Chen.  On Asymptotic Normality of Limiting Density Functions with Bayesian Implications , 1985 .

[45]  Yuichi Kitamura,et al.  Empirical Likelihood Methods in Econometrics: Theory and Practice , 2006 .

[46]  Whitney K. Newey,et al.  Higher Order Properties of Gmm and Generalized Empirical Likelihood Estimators , 2003 .

[47]  R. Kress.  Linear Integral Equations , 1989 .

[48]  Yum K. Kwan.  Asymptotic Bayesian analysis based on a limited information estimator , 1999 .

[49]  Jae-Young Kim,et al.  Limited information likelihood and Bayesian analysis , 2002 .

[50]  T. Ferguson.  Prior Distributions on Spaces of Probability Measures , 1974 .

[51]  N. Hjort,et al.  Exact Inference for Random Dirichlet Means , 2005 .

[52]  Richard J. Smith.  Alternative Semi-parametric Likelihood Approaches to Generalised Method of Moments Estimation , 1997 .

[53]  Lan Peter Hamen,et al.  Generalized Instrumental Variables Estimation of Nonlinear Rational Expectations Models , 1982 .