Parallelization Experience with Four Canonical Econometric Models Using ParMitISEM

This paper presents the parallel computing implementation of the MitISEM algorithm, labeled Parallel MitISEM. The basic MitISEM algorithm, introduced by Hoogerheide, Opschoor and Van Dijk (2012), provides an automatic and flexible method to approximate a non-elliptical target density using adaptive mixtures of Student-t densities, where only a kernel of the target density is required. The approximation can be used as a candidate density in Importance Sampling or Metropolis Hastings methods for Bayesian inference on model parameters and probabilities. We present and discuss four canonical econometric models using a Graphics Processing Unit and a multi-core Central Processing Unit version of the MitISEM algorithm. The results show that the parallelization of the MitISEM algorithm on Graphics Processing Units and multi-core Central Processing Units is straightforward and fast to program using MATLAB. Moreover the speed performance of the Graphics Processing Unit version is much higher than the Central Processing Unit one.

[1]  Jacques H. Dreze,et al.  Bayesian regression analysis using poly-t densities , 1977 .

[2]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[3]  T. Kloek,et al.  Bayesian estimates of equation system parameters, An application of integration by Monte Carlo , 1976 .

[4]  Herb Sutter,et al.  A Fundamental Turn Toward Concurrency in Software , 2008 .

[5]  H. V. Dijk,et al.  Bayesian Near-Boundary Analysis in Basic Macroeconomic Time-Series Models , 2008 .

[6]  Markku Lanne,et al.  Estimation of DSGE Models under Diffuse Priors and Data-Driven Identification Constraints , 2015 .

[7]  Arnaud Doucet,et al.  On the Utility of Graphics Cards to Perform Massively Parallel Simulation of Advanced Monte Carlo Methods , 2009, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[8]  Lennart F. Hoogerheide,et al.  Censored Posterior and Predictive Likelihood in Bayesian Left-Tail Prediction for Accurate Value at Risk Estimation , 2013 .

[9]  Sergei Morozov,et al.  Massively Parallel Computation Using Graphics Processors with Application to Optimal Experimentation in Dynamic Control , 2011, Computational Economics.

[10]  Lennart F. Hoogerheide,et al.  A Class of Adaptive Importance Sampling Weighted EM Algorithms for Efficient and Robust Posterior and Predictive Simulation , 2012 .

[11]  M. Pitt,et al.  Importance Sampling Squared for Bayesian Inference in Latent Variable Models , 2013, 1309.3339.

[12]  Chas. Boyd.  Data-Parallel Computing , 2008, ACM Queue.

[13]  Jesús Fernández-Villaverde,et al.  Tapping the Supercomputer Under Your Desk: Solving Dynamic Equilibrium Models With Graphics Processors , 2010 .

[14]  Lennart F. Hoogerheide,et al.  A comparative study of Monte Carlo methods for efficient evaluation of marginal likelihood , 2012, Comput. Stat. Data Anal..

[15]  Eric M. Aldrich.  GPU Computing in Economics , 2013 .

[16]  Lennart F. Hoogerheide,et al.  The R Package Mitlsem: Mixture of Student-T Distributions Using Importance Sampling Weighted Expectation Maximization for Efficient and Robust Simulation , 2012 .

[17]  Lennart F. Hoogerheide,et al.  Joint Independent Metropolis-Hastings Methods for Nonlinear Non-Gaussian State Space Models , 2012 .

[18]  Herb Sutter,et al.  The Free Lunch Is Over A Fundamental Turn Toward Concurrency in Software , 2013 .

[19]  H. V. Dijk,et al.  On the Rise of Bayesian Econometrics after Cowles Foundation Monographs 10, 14 , 2014 .

[20]  Lennart F. Hoogerheide,et al.  Bayesian Analysis of Instrumental Variable Models: Acceptance-Rejection within Direct Monte Carlo , 2012 .

[21]  Frank Arens,et al.  [Welcome to the Jungle]. , 2014, Pflege Zeitschrift.

[22]  Jie Cheng,et al.  Programming Massively Parallel Processors. A Hands-on Approach , 2010, Scalable Comput. Pract. Exp..

[23]  Markku Lanne,et al.  Noncausal Bayesian Vector Autoregression , 2014 .

[24]  Lennart F. Hoogerheide,et al.  On the Shape of Posterior Densities and Credible Sets in Instrumental Variable Regression Models with Reduced Rank: An Application of Flexible Sampling Methods Using Neural Networks , 2005 .

[25]  P. Prescott,et al.  Monte Carlo Methods , 1964, Computational Statistical Physics.

[26]  Jean-Michel Marin,et al.  Adaptive importance sampling in general mixture classes , 2007, Stat. Comput..

[27]  Eric Zivot,et al.  Practical Issues in the Analysis of Univariate GARCH Models , 2009 .

[28]  Christopher Holmes,et al.  Some of the What?, Why?, How?, Who? and Where? of Graphics Processing Unit Computing for Bayesian Analysis , 2011 .

[29]  J. Geweke,et al.  Bayesian Inference in Econometric Models Using Monte Carlo Integration , 1989 .

[30]  Chao Yang,et al.  Learn From Thy Neighbor: Parallel-Chain and Regional Adaptive MCMC , 2009 .

[31]  Stefano Grassi,et al.  Heterogeneous Computing in Economics: A Simplified Approach , 2013, Computational Economics.

[32]  David Card,et al.  Using Geographic Variation in College Proximity to Estimate the Return to Schooling , 1993 .

[33]  Radu V. Craiu,et al.  Multiprocess parallel antithetic coupling for backward and forward Markov Chain Monte Carlo , 2005, math/0505631.

[34]  Christian P. Robert,et al.  Using Parallel Computation to Improve Independent Metropolis–Hastings Based Estimation , 2010, ArXiv.

[35]  M. Tanner,et al.  Facilitating the Gibbs Sampler: The Gibbs Stopper and the Griddy-Gibbs Sampler , 1992 .

[36]  Herman K. van Dijk,et al.  An algorithm for the computation of posterior moments and densities using simple importance sampling , 1986 .

[37]  Rémi Bardenet,et al.  Monte Carlo Methods , 2013, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[38]  A. Zellner.  An Introduction to Bayesian Inference in Econometrics , 1971 .

[39]  Arnold Zellner,et al.  An Introduction to Bayesian Inference in Econometrics. , 1974 .

[40]  Michael P. Murray,et al.  Instrumental Variables , 2011, International Encyclopedia of Statistical Science.

[41]  ECONOMETRICS: MATLAB toolbox of econometrics functions , 1998 .

[42]  Sally Rosenthal,et al.  Parallel computing and Monte Carlo algorithms , 1999 .

[43]  Herman K. van Dijk,et al.  Posterior-Predictive Evidence on US Inflation Using Extended New Keynesian Phillips Curve Models with Non-Filtered Data , 2013 .

[44]  Lennart F. Hoogerheide,et al.  Natural conjugate priors for the instrumental variables regression model applied to the Angrist–Krueger data , 2006 .

[45]  T. Bollerslev,et al.  Generalized autoregressive conditional heteroskedasticity , 1986 .

[46]  Xiao-Li Meng,et al.  A Note on Bivariate Distributions That are Conditionally Normal , 1991 .

[47]  Jacques H. Dreze,et al.  Bayesian Limited Information Analysis of the Simultaneous Equations Model , 1976 .

[48]  John Geweke,et al.  Massively Parallel Sequential Monte Carlo for Bayesian Inference , 2011 .

[49]  Frank Kleibergen,et al.  BAYESIAN SIMULTANEOUS EQUATIONS ANALYSIS USING REDUCED RANK STRUCTURES , 1998, Econometric Theory.