A Practical Guide to Parallelization in Economics

This guide provides a practical introduction to parallel computing in economics. After a brief introduction to the basic ideas of parallelization, we show how to parallelize a prototypical application in economics using, on CPUs, Julia, Matlab, R, Python, C++-OpenMP, Rcpp–OpenMP, and C++-MPI, and, on GPUs, CUDA and OpenACC. We provide code that the user can download and fork, present comparative results, and explain the strengths and weaknesses of each approach. We conclude with some additional remarks about alternative approaches. Institutional subscribers to the NBER working paper series, and residents of developing countries may download this paper without additional charge at www.nber.org.

[1]  Jukka Corander,et al.  Orthogonal parallel MCMC methods for sampling and optimization , 2015, Digit. Signal Process..

[2]  Jie Cheng,et al.  Programming Massively Parallel Processors. A Hands-on Approach , 2010, Scalable Comput. Pract. Exp..

[3]  Christian de Schryver,et al.  FPGA Based Accelerators for Financial Applications , 2015 .

[4]  Christian P. Robert,et al.  The Bayesian choice : from decision-theoretic foundations to computational implementation , 2007 .

[5]  Charles J. Murray.  The Supermen: The Story of Seymour Cray and the Technical Wizards Behind the Supercomputer , 1997 .

[6]  Rob Farber,et al.  Parallel Programming with OpenACC , 2016 .

[7]  David Zarruk Valencia.  Wall Street or Main Street : Who to Bail Out ? , 2017 .

[8]  A. Gelman,et al.  Weak convergence and optimal scaling of random walk Metropolis algorithms , 1997 .

[9]  Dirk Krueger,et al.  Social Security Reform with Heterogeneous Agents , 1999 .

[10]  L. Devroye.  Non-Uniform Random Variate Generation , 1986 .

[11]  Kent A. Smetters,et al.  Analyzing fiscal policies in a heterogeneous-agent overlapping-generations economy , 2014 .

[12]  Y. Ben-Porath.  The Production of Human Capital and the Life Cycle of Earnings , 1967, Journal of Political Economy.

[13]  Ben Calderhead,et al.  A general construction for parallelizing Metropolis−Hastings algorithms , 2014, Proceedings of the National Academy of Sciences.

[14]  Ingvar Strid.  Efficient parallelisation of Metropolis-Hastings algorithms using a prefetching approach , 2010, Comput. Stat. Data Anal..

[15]  Pascal J. Maenhout,et al.  Consumption and Portfolio Choice over the Life Cycle , 2005 .

[16]  Arch D. Robison,et al.  Structured Parallel Programming: Patterns for Efficient Computation , 2012 .

[17]  Jesús Fernández-Villaverde,et al.  CONSUMPTION AND SAVING OVER THE LIFE CYCLE: HOW IMPORTANT ARE CONSUMER DURABLES? , 2011, Macroeconomic Dynamics.

[18]  Jesús Fernández-Villaverde,et al.  Tapping the Supercomputer Under Your Desk: Solving Dynamic Equilibrium Models With Graphics Processors , 2010 .

[19]  G.E. Moore,et al.  Cramming More Components Onto Integrated Circuits , 1998, Proceedings of the IEEE.

[20]  Eric M. Aldrich.  GPU Computing in Economics , 2013 .

[21]  Thomas Rauber,et al.  Parallel Programming: for Multicore and Cluster Systems , 2010, Parallel Programming, 3rd Ed..

[22]  Christian Gourieroux,et al.  Simulation-based econometric methods , 1996 .

[23]  Kenneth Flamm.  Measuring Moore's Law: Evidence from Price, Cost, and Quality Indexes , 2018 .

[24]  Christina Freytag,et al.  Using Mpi Portable Parallel Programming With The Message Passing Interface , 2016 .

[25]  Adrian Jackson,et al.  OpenACC for Programmers: Concepts and Strategies , 2017 .

[26]  Gerhard Wellein,et al.  Introduction to High Performance Computing for Scientists and Engineers , 2010, Chapman and Hall / CRC computational science series.

[27]  Lawrence Snyder,et al.  Principles of Parallel Programming , 2008 .

[28]  Jesús Fernández-Villaverde,et al.  A Comparison of Programming Languages in Economics , 2015 .

[29]  David Kaeli,et al.  Introduction to Parallel Programming , 2013 .