Parallel random numbers: As easy as 1, 2, 3

Most pseudorandom number generators (PRNGs) scale poorly to massively parallel high-performance computation because they are designed as sequentially dependent state transformations. We demonstrate that independent, keyed transformations of counters produce a large alternative class of PRNGs with excellent statistical properties (long period, no discernable structure or correlation). These counter-based PRNGs are ideally suited to modern multi- core CPUs, GPUs, clusters, and special-purpose hardware because they vectorize and parallelize well, and require little or no memory for state. We introduce several counter-based PRNGs: some based on cryptographic standards (AES, Threefish) and some completely new (Philox). All our PRNGs pass rigorous statistical tests (including TestUOl's BigCrush) and produce at least 264 unique parallel streams of random numbers, each with period 2128 or more. In addition to essentially unlimited parallel scalability, our PRNGs offer excellent single-chip performance: Philox is faster than the CURAND library on a single NVIDIA GPU.

[1]  Claude E. Shannon,et al.  Communication theory of secrecy systems , 1949, Bell Syst. Tech. J..

[2]  T. E. Hull,et al.  Random Number Generators , 1962 .

[3]  E. McKinney.  Generalized Birthday Problem , 1966 .

[4]  H. Feistel.  Cryptography and Computer Privacy , 1973 .

[5]  Donald E. Knuth,et al.  The art of computer programming. Vol.2: Seminumerical algorithms , 1981 .

[6]  M. Kalos,et al.  Monte Carlo methods , 1986 .

[7]  Ralph Howard,et al.  Data encryption standard , 1987 .

[8]  G. C. Fox,et al.  Solving Problems on Concurrent Processors , 1988 .

[9]  A. Matteis,et al.  Parallelization of random number generators and long-range correlations , 1988 .

[10]  S. K. Park,et al.  Random number generators: good ones are hard to find , 1988, CACM.

[11]  A. De Matteis,et al.  Long-range correlations in linear and nonlinear random number generators , 1990, Parallel Comput..

[12]  Alan M. Ferrenberg,et al.  Monte Carlo simulations: Hidden errors from "good" random number generators. , 1992, Physical review letters.

[13]  Pierre L'Ecuyer,et al.  A search for good multiple recursive random number generators , 1993, TOMC.

[14]  Pierre L'Ecuyer,et al.  Combined Multiple Recursive Random Number Generators , 1995, Oper. Res..

[15]  Paul Coddington,et al.  Random Number Generators for Parallel Computers , 1997 .

[16]  Takuji Nishimura,et al.  Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator , 1998, TOMC.

[17]  Peter Hellekalek.  Don't trust parallel Monte Carlo! , 1998, Workshop on Parallel and Distributed Simulation.

[18]  P. Hellekalek.  Good random number generators are (not so) easy to find , 1998 .

[19]  Elaine B. Barker,et al.  A Statistical Test Suite for Random and Pseudorandom Number Generators for Cryptographic Applications , 2000 .

[20]  Michael Mascagni,et al.  Algorithm 806: SPRNG: a scalable library for pseudorandom number generation , 1999, TOMS.

[21]  Morris J. Dworkin,et al.  Recommendation for Block Cipher Modes of Operation: Methods and Techniques , 2001 .

[22]  Morris J. Dworkin,et al.  SP 800-38A 2001 edition. Recommendation for Block Cipher Modes of Operation: Methods and Techniques , 2001 .

[23]  David Thomas,et al.  The Art in Computer Programming , 2001 .

[24]  William H. Press,et al.  Numerical recipes in C , 2002 .

[25]  Peter Hellekalek,et al.  Empirical evidence concerning AES , 2003, TOMC.

[26]  Pierre L'Ecuyer,et al.  Improved long-period generators based on linear recurrences modulo 2 , 2004, TOMS.

[27]  Yehuda Lindell,et al.  Introduction to Modern Cryptography , 2004 .

[28]  Makoto Matsumoto,et al.  Common defects in initialization of pseudorandom number generators , 2007, TOMC.

[29]  Pierre L'Ecuyer,et al.  TestU01: A C library for empirical testing of random number generators , 2006, TOMS.

[30]  Peter L. Freddolino,et al.  Vulnerability in Popular Molecular Dynamics Packages Concerning Langevin and Andersen Dynamics. , 2008, Journal of chemical theory and computation.

[31]  Li-Yi Wei,et al.  Parallel white noise generation on a GPU via cryptographic hash , 2008, I3D '08.

[32]  Peter Schwabe,et al.  New AES Software Speed Records , 2008, INDOCRYPT.

[33]  Stefan Lucks,et al.  The Skein Hash Function Family , 2009 .

[34]  Daniel J. Sindhikara,et al.  Bad Seeds Sprout Perilous Dynamics: Stochastic Thermostat Induced Trajectory Synchronization in Biomolecules. , 2009, Journal of chemical theory and computation.

[35]  J. P. Grossman,et al.  Millisecond-scale molecular dynamics simulations on Anton , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[36]  Simon Heron,et al.  Encryption: Advanced Encryption Standard (AES) , 2009 .

[37]  Marc Olano,et al.  GPU random numbers via the tiny encryption algorithm , 2010, HPG '10.

[38]  Richard P. Brent,et al.  Some long-period random number generators using shifts and xors , 2007, ArXiv.

[39]  Joseph A. Bank,et al.  Supporting Online Material Materials and Methods Figs. S1 to S10 Table S1 References Movies S1 to S3 Atomic-level Characterization of the Structural Dynamics of Proteins , 2022 .

[40]  Morris J. Dworkin,et al.  Recommendation for Block Cipher Modes of Operation: Methods for Key Wrapping , 2012 .

[41]  P. L’Ecuyer.  Random Number Generation , 2012 .

[42]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.