Practical distribution of random streams for stochastic High Performance Computing

Random number generators are necessary in every simulation which includes stochastic aspects. For High Performance Computing, there is an increasing interest in the distribution of parallel random number streams. Even if we have now at our disposal statistically sound random number generators according to very tough testing libraries, their parallelization can still be a delicate problem. A set of recent publications shows it still has to be mastered by the scientific community. In this paper, we discuss the different partitioning techniques currently in use to provide independent streams with their corresponding software. We try to give an up-to-date state of the art in the parallelization of random numbers for High Performance Computing from the point of view of a simulation practitioner.

[1]  Pierre L'Ecuyer,et al.  rstream: Streams of Random Numbers for Stochastic Simulation , 2005 .

[2]  A. De Matteis,et al.  Long-range correlations in linear and nonlinear random number generators , 1990, Parallel Comput..

[3]  Krzysztof Pawlikowski,et al.  Towards Credible and Fast Quantitative Stochastic Simulation , 2003 .

[4]  P. L’Ecuyer.  Random Number Generation , 2012 .

[5]  A. Matteis,et al.  Parallelization of random number generators and long-range correlations , 1988 .

[6]  Marco Tomassini,et al.  Generating high-quality random numbers in parallel by cellular automata , 1999, Future Gener. Comput. Syst..

[7]  Paul D. Coddington,et al.  Techniques for empirical testing of parallel random number generators , 1998, ICS '98.

[8]  Michael Mascagni.  Some Methods of Parallel Pseudorandom Number Generation , 1999 .

[9]  Pierre L'Ecuyer,et al.  Pseudorandom Number Generators , 2010 .

[10]  Paul Coddington,et al.  Random Number Generators for Parallel Computers , 1997 .

[11]  Krzysztof Pawlikowski,et al.  Do Not Trust All Simulation Studies of Telecommunication Networks , 2003, ICOIN.

[12]  Mario Rütti,et al.  A Random Number Generator Test Suite for the C++ Standard , 2004 .

[13]  Takuji Nishimura,et al.  Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator , 1998, TOMC.

[14]  Pierre L'Ecuyer,et al.  TestU01: A C library for empirical testing of random number generators , 2006, TOMS.

[15]  Yaohang Li,et al.  Improving performance via computational replication on a large-scale computational grid , 2003, CCGrid 2003. 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid, 2003. Proceedings..

[16]  Burton J. Smith,et al.  Pseudo-random trees in Monte Carlo , 1984, Parallel Comput..

[17]  Donald Ervin Knuth,et al.  The Art of Computer Programming , 1968 .

[18]  Kuo-Chan Huang,et al.  Parallel use of multiplicative congruential random number generators , 2006, Comput. Phys. Commun..

[19]  Peter Hellekalek.  Don't trust parallel Monte Carlo! , 1998, Workshop on Parallel and Distributed Simulation.

[20]  P. Hellekalek.  Good random number generators are (not so) easy to find , 1998 .

[21]  William F. Eddy.  Random number generators for parallel processors , 1990 .

[22]  Marco Tomassini,et al.  Generating Parallel Random Number Generators By Cellular Programming , 1996 .

[23]  Jürgen Eichenauer-Herrmann,et al.  A remark on long-range correlations in multiplicative congruential pseudo random number generators , 1989 .

[24]  Andreas Uhl,et al.  Parallel Random Number Generation: Long-Range Correlations Among Multiple Processors , 1999, ACPC.

[25]  Makoto Matsumoto,et al.  SIMD-Oriented Fast Mersenne Twister: a 128-bit Pseudorandom Number Generator , 2008 .

[26]  B. Hechenleitner,et al.  On Shortcomings of the ns-2 Random Number Generator , 2002 .

[27]  A. De Matteis,et al.  Controlling Correlations in Parallel Monte Carlo , 1995, Parallel Comput..

[28]  V. Breton,et al.  Rigorous Distribution of Stochastic Simulations Using the DistMe Toolkit , 2008, IEEE Transactions on Nuclear Science.

[29]  Bernhard Hechenleitner,et al.  Pitfalls when using parallel streams in OMNeT + + simulations , 2003 .

[30]  A. De Matteis,et al.  A class of parallel random number generators , 1990, Parallel Comput..

[31]  Romain Reuillon.  Simulations stochastiques en environnements distribués. Application aux grilles de calcul. (Stochastic simulations in distributed environments. Applications to computational grids) , 2008 .

[32]  Pierre L'Ecuyer,et al.  Improved long-period generators based on linear recurrences modulo 2 , 2004, TOMS.

[33]  Paul D. Coddington,et al.  JAPARA - a Java parallel random number generator library for high-performance computing , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[34]  Michael Mascagni,et al.  Parameterizing parallel multiplicative lagged-Fibonacci generators , 2004, Parallel Comput..

[35]  Michael Mascagni.  SPRNG: A Scalable Library for Pseudorandom Number Generation , 1999, PPSC.

[36]  Pierre L'Ecuyer,et al.  An Object-Oriented Random-Number Package with Many Long Streams and Substreams , 2002, Oper. Res..

[37]  F. James.  A Review of Pseudorandom Number Generators , 1990 .

[38]  Michael Mascagni,et al.  Testing parallel random number generators , 2003, Parallel Comput..

[39]  Krzysztof Pawlikowski,et al.  On credibility of simulation studies of telecommunication networks , 2002, IEEE Commun. Mag..

[40]  Jerome Spanier,et al.  Dynamic creation of pseudorandom number generators , 2000 .

[41]  R.M. Fujimoto,et al.  Parallel and distributed simulation systems , 2001, Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304).

[42]  Matthias Troyer,et al.  A Generic Random Number Generator Test Suite , 2004 .

[43]  Hongmei Chi,et al.  Parallel linear congruential generators with Sophie-Germain moduli , 2004, Parallel Comput..

[44]  M. Mascagni.  High-Performance Monte Carlo Tools [Conferences & Workshops] , 1998 .

[45]  M. J. Durst.  Using linear congruential generators for parallel random number generation , 1989, WSC '89.

[46]  David R.C. Hill,et al.  URNG: A portable optimization technique for software applications requiring pseudo-random numbers , 2003, Simul. Model. Pract. Theory.

[47]  Stephan Mertens,et al.  Random numbers for large scale distributed Monte Carlo simulations , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[48]  I. Buvat,et al.  Fully 3D Monte Carlo image reconstruction in SPECT using functional regions , 2005, physics/0510245.

[49]  Pierre L'Ecuyer,et al.  Efficient Jump Ahead for 2-Linear Random Number Generators , 2006, INFORMS J. Comput..

[50]  A. De Matteis,et al.  Computation of critical distances within multiplicative congruential pseudorandom number sequences , 1992 .

[51]  Michael Mascagni,et al.  Corrigendum: Algorithm 806: SPRNG: a scalable library for pseudorandom number generation , 2000, TOMS.

[52]  Pierre L’Ecuyer,et al.  Random Number Generation , 2008, Encyclopedia of Algorithms.

[53]  Pierre L'Ecuyer,et al.  Efficient Jump Ahead for (F-openface) 2 -Linear Random Number Generators , 2008 .

[54]  David R. C. Hill,et al.  Parallelization Of Monte Carlo Simulations And Submission To A Grid Environment , 2004, Parallel Process. Lett..

[55]  I Buvat,et al.  Fully 3D Monte Carlo reconstruction in SPECT: a feasibility study , 2005, Physics in medicine and biology.

[56]  Pierre L'Ecuyer,et al.  Simulation in Java with SSJ , 2005, Proceedings of the Winter Simulation Conference, 2005..

[57]  Ora E. Percus,et al.  Random Number Generators for MIMD Parallel Processors , 1989, J. Parallel Distributed Comput..

[58]  Pierre L'Ecuyer,et al.  Software for uniform random number generation: distinguishing the good and the bad , 2001, Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304).

[59]  J. McCaskill,et al.  Parallel random number generator for inexpensive configurable hardware cells , 2001 .

[60]  K Entacher,et al.  Linear Congruential Generators for Parallel Monte Carlo: the Leap-Frog Case. , 1998, Monte Carlo Methods Appl..

[61]  J. Kleijnen.  Statistical tools for simulation practitioners , 1986 .