Machine Learning Econometrics: Bayesian Algorithms and Methods

As the amount of economic and other data generated worldwide increases vastly, a challenge for future generations of econometricians will be to master efficient algorithms for inference in empirical models with large information sets. This Chapter provides a review of popular estimation algorithms for Bayesian inference in econometrics and surveys alternative algorithms developed in machine learning and computing science that allow for efficient computation in high-dimensional settings. The focus is on scalability and parallelizability of each algorithm, as well as their ability to be adopted in various empirical settings in economics and finance.

[1]  Arnaud Doucet,et al.  On Markov chain Monte Carlo methods for tall data , 2015, J. Mach. Learn. Res..

[2]  Xiangyu Wang,et al.  Parallelizing MCMC via Weierstrass Sampler , 2013, 1312.4605.

[3]  Aart J. C. Bik,et al.  Pregel: a system for large-scale graph processing , 2010, SIGMOD Conference.

[4]  Dimitris Korobilis,et al.  Adaptive hierarchical priors for high-dimensional vector autoregressions , 2019 .

[5]  M. Tanner,et al.  Facilitating the Gibbs Sampler: The Gibbs Stopper and the Griddy-Gibbs Sampler , 1992 .

[6]  Dimitris Korobilis,et al.  High-Dimensional Macroeconomic Forecasting Using Message Passing Algorithms , 2019, Journal of Business & Economic Statistics.

[7]  S. A. Sisson,et al.  Overview of Approximate Bayesian Computation , 2018, 1802.09720.

[8]  D. Lindley,et al.  Approximate Bayesian methods , 1980 .

[9]  Ryan P. Adams,et al.  Patterns of Scalable Bayesian Inference , 2016, Found. Trends Mach. Learn..

[10]  Jun Zhu,et al.  Big Learning with Bayesian Methods , 2014, ArXiv.

[11]  J. Naylor,et al.  Applications of a Method for the Efficient Computation of Posterior Distributions , 1982 .

[12]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001 .

[13]  Veronika Rockova,et al.  EMVS: The EM Approach to Bayesian Variable Selection , 2014 .

[14]  J. Geweke,et al.  Bayesian Inference in Econometric Models Using Monte Carlo Integration , 1989 .

[15]  P. Laplace.  Memoir on the Probability of the Causes of Events , 1986 .

[16]  Chao Yang,et al.  Learn From Thy Neighbor: Parallel-Chain and Regional Adaptive MCMC , 2009 .

[17]  Christian P. Robert,et al.  Bayesian computation: a summary of the current state, and samples backwards and forwards , 2015, Statistics and Computing.

[18]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[19]  Jonathan R Goodman,et al.  Ensemble samplers with affine invariance , 2010 .

[20]  Heikki Haario,et al.  Efficient MCMC for Climate Model Parameter Estimation: Parallel Adaptive Chains and Early Rejection , 2012 .

[21]  G. Koop,et al.  Bayesian Compressed Vector Autoregressions , 2017, Journal of Econometrics.

[22]  David T. Frazier,et al.  Approximate Bayesian forecasting , 2017, International Journal of Forecasting.

[23]  David M. Blei,et al.  Variational Inference: A Review for Statisticians , 2016, ArXiv.

[24]  S. Chib,et al.  Understanding the Metropolis-Hastings Algorithm , 1995 .

[25]  Edward I. George,et al.  Bayes and big data: the consensus Monte Carlo algorithm , 2016, Big Data and Information Theory.

[26]  Carl E. Rasmussen,et al.  Bayesian Monte Carlo , 2002, NIPS.

[27]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[28]  N. Shephard,et al.  Stochastic Volatility: Likelihood Inference And Comparison With Arch Models , 1996 .

[29]  M. Girolami,et al.  Riemann manifold Langevin and Hamiltonian Monte Carlo methods , 2011, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[30]  Xiangyu Wang,et al.  Parallel MCMC via Weierstrass Sampler , 2013, ArXiv.

[31]  Jun S. Liu,et al.  The Multiple-Try Method and Local Optimization in Metropolis Sampling , 2000 .