On the existence of moments for high dimensional importance sampling

Theoretical results for importance sampling rely on the existence of certain moments of the importance weights, which are the ratios between the proposal and target densities. In particular, a finite variance ensures square root convergence and asymptotic normality of the importance sampling estimate, and can be important for the reliability of the method in practice. We derive conditions for the existence of any required moments of the weights for Gaussian proposals and show that these conditions are almost necessary and sufficient for a wide range of models with latent Gaussian components. Important examples are time series and panel data models with measurement densities which belong to the exponential family. We introduce practical and simple methods for checking and imposing the conditions for the existence of the desired moments. We develop a two component mixture proposal that allows us to flexibly adapt a given proposal density into a robust importance density. These methods are illustrated on a wide range of models including generalized linear mixed models, non-Gaussian nonlinear state space models and panel data models with autoregressive random effects.

[1]  S. Koopman,et al.  Numerically Accelerated Importance Sampling for Nonlinear Non-Gaussian State-Space Models , 2012 .

[2]  Francesco Bartolucci,et al.  Latent Markov Models for Longitudinal Data , 2012 .

[3]  Ralph S. Silva,et al.  On Some Properties of Markov Chain Monte Carlo Simulation Methods Based on the Particle Filter , 2012 .

[4]  Y. Saad,et al.  Numerical Methods for Large Eigenvalue Problems , 2011 .

[5]  N. Shephard,et al.  BAYESIAN INFERENCE BASED ONLY ON SIMULATED LIKELIHOOD: PARTICLE FILTER ANALYSIS OF DYNAMIC ECONOMIC MODELS , 2011, Econometric Theory.

[6]  Drew D. Creal,et al.  Testing the assumptions behind importance sampling , 2009 .

[7]  C. Andrieu,et al.  The pseudo-marginal approach for efficient Monte Carlo computations , 2009, 0903.5480.

[8]  Torben G. Andersen,et al.  Stochastic volatility , 2003 .

[9]  L. Bauwens,et al.  Efficient importance sampling for ML estimation of SCD models , 2009, Comput. Stat. Data Anal..

[10]  H. Manner,et al.  Dynamic stochastic copula models: Estimation, inference and applications , 2012 .

[11]  Robert C. Jung,et al.  Dynamic Factor Models for Multivariate Count Data: An Application to Stock-Market Trading Activity , 2008 .

[12]  Donald E. Myers,et al.  Linear and Generalized Linear Mixed Models and Their Applications , 2008, Technometrics.

[13]  S. Koopman,et al.  Monte Carlo estimation for nonlinear non-Gaussian state space models , 2007 .

[14]  J. Richard,et al.  Efficient high-dimensional importance sampling , 2007 .

[15]  Luc Bauwens,et al.  Stochastic Conditional Intensity Processes , 2006 .

[16]  Melvin J. Hinich,et al.  Time Series Analysis by State Space Methods , 2001 .

[17]  G. Molenberghs.  Applied Longitudinal Analysis , 2005 .

[18]  Christian P. Robert,et al.  Monte Carlo Statistical Methods (Springer Texts in Statistics) , 2005 .

[19]  Louis H. Y. Chen,et al.  Normal approximation under local dependence , 2004, math/0410104.

[20]  J. Richard,et al.  Univariate and Multivariate Stochastic Volatility Models: Estimation and Diagnostics , 2003 .

[21]  Tim Hesterberg,et al.  Monte Carlo Strategies in Scientific Computing , 2002, Technometrics.

[22]  Siem Jan Koopman,et al.  A simple and efficient simulation smoother for state space time series analysis , 2002 .

[23]  H. Haario,et al.  An adaptive Metropolis algorithm , 2001 .

[24]  A. Owen,et al.  Safe and Effective Importance Sampling , 2000 .

[25]  Siem Jan Koopman,et al.  Time Series Analysis of Non-Gaussian Observations Based on State Space Models from Both Classical and Bayesian Perspectives , 1999 .

[26]  J. Durbin,et al.  Monte Carlo maximum likelihood estimation for non-Gaussian state space models , 1997 .

[27]  M. Pitt,et al.  Likelihood analysis of non-Gaussian measurement time series , 1997 .

[28]  GourierouxMonfort.  Statistics and Econometric Models, Volume 2 , 1996 .

[29]  A. Harvey,et al.  5 Stochastic volatility , 1996 .

[30]  N. Shephard,et al.  The simulation smoother for time series models , 1995 .

[31]  Christian Gourieroux,et al.  Statistics and econometric models , 1995 .

[32]  James O. Berger,et al.  Noninformative Priors and Bayesian Testing for the AR(1) Model , 1994, Econometric Theory.

[33]  J. Geweke,et al.  Bayesian Inference in Econometric Models Using Monte Carlo Integration , 1989 .

[34]  A. C. Berry.  The accuracy of the Gaussian approximation to the sum of independent variates , 1941 .

[35]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .