Distribution of random streams for simulation practitioners

There is an increasing interest in the distribution of parallel random number streams in the high‐performance computing community particularly, with the manycore shift. Even if we have at our disposal statistically sound random number generators according to the latest and thorough testing libraries, their parallelization can still be a delicate problem. Indeed, a set of recent publications shows it still has to be mastered by the scientific community. With the arrival of multi‐core and manycore processor architectures on the scientist desktop, modelers who are non‐specialists in parallelizing stochastic simulations need help and advice in distributing rigorously their experimental plans and replications according to the state of the art in pseudo‐random numbers parallelization techniques. In this paper, we discuss the different partitioning techniques currently in use to provide independent streams with their corresponding software. In addition to the classical approaches in use to parallelize stochastic simulations on regular processors, this paper also presents recent advances in pseudo‐random number generation for general‐purpose graphical processing units. The state of the art given in this paper is written for simulation practitioners. Copyright © 2012 John Wiley & Sons, Ltd.

[1]  A. De Matteis,et al.  A class of parallel random number generators , 1990, Parallel Comput..

[2]  M. Sipper,et al.  Generating Parallel Random Number Generators by Cellular Programming , 1996 .

[3]  Jürgen Eichenauer-Herrmann,et al.  A remark on long-range correlations in multiplicative congruential pseudo random number generators , 1989 .

[4]  Andreas Uhl,et al.  Parallel Random Number Generation: Long-Range Correlations Among Multiple Processors , 1999, ACPC.

[5]  Bernhard Hechenleitner,et al.  Pitfalls when using parallel streams in OMNeT + + simulations , 2003 .

[6]  Krzysztof Pawlikowski,et al.  Towards Credible and Fast Quantitative Stochastic Simulation , 2003 .

[7]  David R. C. Hill,et al.  Reliable Initialization of GPU-enabled Parallel Stochastic Simulations Using Mersenne Twister for Graphics Processors , 2010, ArXiv.

[8]  Makoto Matsumoto,et al.  SIMD-Oriented Fast Mersenne Twister: a 128-bit Pseudorandom Number Generator , 2008 .

[9]  Romain Reuillon.  Simulations stochastiques en environnements distribués. Application aux grilles de calcul. (Stochastic simulations in distributed environments. Applications to computational grids) , 2008 .

[10]  Mario Rütti,et al.  A Random Number Generator Test Suite for the C++ Standard , 2004 .

[11]  David R. C. Hill,et al.  Declarative task delegation in OpenMOLE , 2010, 2010 International Conference on High Performance Computing & Simulation.

[12]  Michael Mascagni,et al.  Corrigendum: Algorithm 806: SPRNG: a scalable library for pseudorandom number generation , 2000, TOMS.

[13]  Burton J. Smith,et al.  Pseudo-random trees in Monte Carlo , 1984, Parallel Comput..

[14]  Wayne H. Enright,et al.  Robust and reliable defect control for Runge-Kutta methods , 2007, TOMS.

[15]  Marco Tomassini,et al.  Generating high-quality random numbers in parallel by cellular automata , 1999, Future Gener. Comput. Syst..

[16]  F. James.  A Review of Pseudorandom Number Generators , 1990 .

[17]  V. Breton,et al.  Rigorous Distribution of Stochastic Simulations Using the DistMe Toolkit , 2008, IEEE Transactions on Nuclear Science.

[18]  Pierre L'Ecuyer,et al.  Improved long-period generators based on linear recurrences modulo 2 , 2004, TOMS.

[19]  Pierre L'Ecuyer,et al.  An Object-Oriented Random-Number Package with Many Long Streams and Substreams , 2002, Oper. Res..

[20]  Michael Mascagni,et al.  Testing parallel random number generators , 2003, Parallel Comput..

[21]  Mark A. Moraes,et al.  Parallel random numbers: As easy as 1, 2, 3 , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[22]  Jason Wittenberg,et al.  Clarify: Software for Interpreting and Presenting Statistical Results , 2003 .

[23]  Srinivas Aluru,et al.  Lagged Fibonacci Random Number Generators for Distributed Memory Parallel Computers , 1997, J. Parallel Distributed Comput..

[24]  Michael B. Giles,et al.  Parallelization Techniques for Random Number Generators , 2011 .

[25]  Rodney Sparapani,et al.  Random Number Generation and Monte Carlo Methods (2nd edition) , 2004 .

[26]  Michael Mascagni.  Some Methods of Parallel Pseudorandom Number Generation , 1999 .

[27]  Pierre L'Ecuyer,et al.  Pseudorandom Number Generators , 2010 .

[28]  Krzysztof Pawlikowski,et al.  On credibility of simulation studies of telecommunication networks , 2002, IEEE Commun. Mag..

[29]  Stephan Mertens,et al.  Random numbers for large scale distributed Monte Carlo simulations , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[30]  Jerome Spanier,et al.  Dynamic creation of pseudorandom number generators , 2000 .

[31]  R.M. Fujimoto,et al.  Parallel and distributed simulation systems , 2001, Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304).

[32]  Matthias Troyer,et al.  A Generic Random Number Generator Test Suite , 2004 .

[33]  Srinivas Aluru,et al.  Fast, Processor-Cardinality Agnostic PRNG with a Tracking Application , 2008, 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing.

[34]  Pierre L'Ecuyer,et al.  rstream: Streams of Random Numbers for Stochastic Simulation , 2005 .

[35]  G. Marsaglia,et al.  A New Class of Random Number Generators , 1991 .

[36]  David R. C. Hill,et al.  Parallelization Of Monte Carlo Simulations And Submission To A Grid Environment , 2004, Parallel Process. Lett..

[37]  J. McCaskill,et al.  Parallel random number generator for inexpensive configurable hardware cells , 2001 .

[38]  I Buvat,et al.  Fully 3D Monte Carlo reconstruction in SPECT: a feasibility study , 2005, Physics in medicine and biology.

[39]  Hongmei Chi,et al.  Parallel linear congruential generators with Sophie-Germain moduli , 2004, Parallel Comput..

[40]  M. Mascagni.  High-Performance Monte Carlo Tools [Conferences & Workshops] , 1998 .

[41]  Paul D. Coddington,et al.  JAPARA - a Java parallel random number generator library for high-performance computing , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[42]  M. J. Durst.  Using linear congruential generators for parallel random number generation , 1989, WSC '89.

[43]  David R.C. Hill,et al.  URNG: A portable optimization technique for software applications requiring pseudo-random numbers , 2003, Simul. Model. Pract. Theory.

[44]  Michael Mascagni,et al.  Parameterizing parallel multiplicative lagged-Fibonacci generators , 2004, Parallel Comput..

[45]  Pierre L’Ecuyer,et al.  Random Number Generation , 2008, Encyclopedia of Algorithms.

[46]  Kuo-Chan Huang,et al.  Parallel use of multiplicative congruential random number generators , 2006, Comput. Phys. Commun..

[47]  P. Hellekalek.  Good random number generators are (not so) easy to find , 1998 .

[48]  Michael Mascagni.  SPRNG: A Scalable Library for Pseudorandom Number Generation , 1999, PPSC.

[49]  William F. Eddy.  Random number generators for parallel processors , 1990 .

[50]  Paul Coddington,et al.  Random Number Generators for Parallel Computers , 1997 .

[51]  Krzysztof Pawlikowski,et al.  Do Not Trust All Simulation Studies of Telecommunication Networks , 2003, ICOIN.

[52]  A. Matteis,et al.  Parallelization of random number generators and long-range correlations , 1988 .

[53]  David R. C. Hill,et al.  Parallel stochastic simulations with rigorous distribution of pseudo‐random numbers with DistMe: Application to life science simulations , 2012, Concurr. Comput. Pract. Exp..

[54]  Harald Niederreiter,et al.  Random number generation and Quasi-Monte Carlo methods , 1992, CBMS-NSF regional conference series in applied mathematics.

[55]  Pierre L'Ecuyer,et al.  TestU01: A C library for empirical testing of random number generators , 2006, TOMS.

[56]  Yaohang Li,et al.  Improving performance via computational replication on a large-scale computational grid , 2003, CCGrid 2003. 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid, 2003. Proceedings..

[57]  Craig B. Borkowf,et al.  Random Number Generation and Monte Carlo Methods , 2000, Technometrics.

[58]  Peter Hellekalek.  Don't trust parallel Monte Carlo! , 1998, Workshop on Parallel and Distributed Simulation.

[59]  Pierre L'Ecuyer,et al.  Software for uniform random number generation: distinguishing the good and the bad , 2001, Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304).

[60]  Gregory Ewing,et al.  Akaroa-2: Exploiting Network Computing by Distributing Stochastic Simulation , 1999 .

[61]  K Entacher,et al.  Linear Congruential Generators for Parallel Monte Carlo: the Leap-Frog Case. , 1998, Monte Carlo Methods Appl..

[62]  J. Kleijnen.  Statistical tools for simulation practitioners , 1986 .

[63]  Takuji Nishimura,et al.  Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator , 1998, TOMC.

[64]  Pierre L'Ecuyer,et al.  Maximally equidistributed combined Tausworthe generators , 1996, Math. Comput..

[65]  George Marsaglia,et al.  A random number generator for PC's , 1990 .

[66]  Francois Panneton.  Construction d'ensembles de points basee sur des recurrences lineaires dans un corps fini de caracteristique 2 pour la simulation Monte Carlo et l'integration quasi-Monte Carlo , 2004 .

[67]  Ora E. Percus,et al.  Random Number Generators for MIMD Parallel Processors , 1989, J. Parallel Distributed Comput..

[68]  Karolj Skala,et al.  Quantum Random Bit Generator Service for Monte Carlo and Other Stochastic Simulations , 2007, LSSC.

[69]  Paul D. Coddington,et al.  Techniques for empirical testing of parallel random number generators , 1998, ICS '98.

[70]  Pierre L'Ecuyer,et al.  Simulation in Java with SSJ , 2005, Proceedings of the Winter Simulation Conference, 2005..

[71]  B. Hechenleitner,et al.  On Shortcomings of the ns-2 Random Number Generator , 2002 .

[72]  A. De Matteis,et al.  Controlling Correlations in Parallel Monte Carlo , 1995, Parallel Comput..

[73]  I. Buvat,et al.  Fully 3D Monte Carlo image reconstruction in SPECT using functional regions , 2005, physics/0510245.

[74]  Pierre L'Ecuyer,et al.  Efficient Jump Ahead for 2-Linear Random Number Generators , 2006, INFORMS J. Comput..

[75]  A. De Matteis,et al.  Computation of critical distances within multiplicative congruential pseudorandom number sequences , 1992 .