TestU01: A C library for empirical testing of random number generators

We introduce TestU01, a software library implemented in the ANSI C language, and offering a collection of utilities for the empirical statistical testing of uniform random number generators (RNGs). It provides general implementations of the classical statistical tests for RNGs, as well as several others tests proposed in the literature, and some original ones. Predefined tests suites for sequences of uniform random numbers over the interval (0, 1) and for bit sequences are available. Tools are also offered to perform systematic studies of the interaction between a specific test and the structure of the point sets produced by a given family of RNGs. That is, for a given kind of test and a given class of RNGs, to determine how large should be the sample size of the test, as a function of the generator's period length, before the generator starts to fail the test systematically. Finally, the library provides various types of generators implemented in generic form, as well as many specific generators proposed in the literature or found in widely used software. The tests can be applied to instances of the generators predefined in the library, or to user-defined generators, or to streams of random numbers produced by any kind of device or stored in files. Besides introducing TestU01, the article provides a survey and a classification of statistical tests for RNGs. It also applies batteries of tests to a long list of widely used RNGs.

[1]  Pierre L'Ecuyer,et al.  On the Deng-Lin random number generators and related methods , 2004, Stat. Comput..

[2]  Pierre L'Ecuyer,et al.  Random Number Generators: Selection Criteria and Testing , 1998 .

[3]  G. Marsaglia,et al.  A New Class of Random Number Generators , 1991 .

[4]  Ramesh C. Agarwal,et al.  Fast pseudorandom-number generators with modulus 2k or 2k - 1 using fused multiply-ad , 2002, IBM J. Res. Dev..

[5]  Pierre L’Ecuyer,et al.  Random Number Generation , 2008, Encyclopedia of Algorithms.

[6]  Pierre L'Ecuyer,et al.  Software for uniform random number generation: distinguishing the good and the bad , 2001, Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304).

[7]  F. B. Brown,et al.  The MCNP5 Random number generator , 2002 .

[8]  Pierre L'Ecuyer,et al.  Improved long-period generators based on linear recurrences modulo 2 , 2004, TOMS.

[9]  L. R. Moore,et al.  An Exhaustive Analysis of Multiplicative Congruential Random Number Generators with Modulus $2^{31} - 1$ , 1986 .

[10]  Pierre L'Ecuyer,et al.  An Object-Oriented Random-Number Package with Many Long Streams and Substreams , 2002, Oper. Res..

[11]  T. W. Anderson,et al.  Asymptotic Theory of Certain "Goodness of Fit" Criteria Based on Stochastic Processes , 1952 .

[12]  P. Whitlock,et al.  A new empirical test for parallel pseudo-random number generators , 2001 .

[13]  Robert E. Greenwood.  Coupon collector’s test for random digits , 1955 .

[14]  William H. Press,et al.  Numerical recipes in C , 2002 .

[15]  Shu Tezuka,et al.  Uniform Random Numbers , 1995 .

[16]  Jean-François Cordeau,et al.  Close-Point Spatial Tests and Their Application to Random Number Generators , 2000, Oper. Res..

[17]  George Marsaglia,et al.  A random number generator for PC's , 1990 .

[18]  Gordana Ugrin-Sparac,et al.  Stochastic investigations of pseudo-random number generators , 1991, Computing.

[19]  T. Ala‐Nissila,et al.  Physical models as tests of randomness. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[20]  M. Z. Wang.  Linear Complexity Profiles and Jump Complexity , 1997, Inf. Process. Lett..

[21]  Jürgen Eichenauer-Herrmann,et al.  Inversive congruential pseudorandom numbers : a tutorial , 1992 .

[22]  P. A. W. Lewis,et al.  A Pseudo-Random Number Generator for the System/360 , 1969, IBM Syst. J..

[23]  Dennis K. J. Lin,et al.  Random Number Generation for the New Century , 2000 .

[24]  Takuji Nishimura,et al.  Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator , 1998, TOMC.

[25]  Cleve B. Moler,et al.  Numerical computing with MATLAB , 2004 .

[26]  Pierre L'Ecuyer,et al.  Maximally equidistributed combined Tausworthe generators , 1996, Math. Comput..

[27]  Keizo Takashima.  LAST VISIT TIME TESTS FOR PSEUDORANDOM NUMBERS , 1996 .

[28]  I. D. Hill,et al.  An Efficient and Portable Pseudo‐Random Number Generator , 1982 .

[29]  G. Marsaglia.  The Structure of Linear Congruential Sequences , 1972 .

[30]  F. James,et al.  RANLUX: A Fortran implementation of the high-quality pseudorandom number generator of Lüscher , 1994 .

[31]  Pierre L'Ecuyer,et al.  Bad Lattice Structures for Vectors of Nonsuccessive Values Produced by Some Linear Recurrences , 1997, INFORMS J. Comput..

[32]  J. P. R. Tootill,et al.  An Asymptotically Random Tausworthe Sequence , 1973, JACM.

[33]  Jürgen Lehn,et al.  A non-linear congruential pseudo random number generator , 1986 .

[34]  D. Darling,et al.  On the Theorems of Kolmogorov-Smirnov , 1960 .

[35]  François Koeune.  Pseudo-random number generator , 2005, Encyclopedia of Cryptography and Security.

[36]  I. Good.  The serial test for sampling numbers and other tests for randomness , 1953, Mathematical Proceedings of the Cambridge Philosophical Society.

[37]  Pierre L'Ecuyer,et al.  Simulation in Java with SSJ , 2005, Proceedings of the Winter Simulation Conference, 2005..

[38]  Pierre L'Ecuyer,et al.  Sparse Serial Tests of Uniformity for Random Number Generators , 1998, SIAM J. Sci. Comput..

[39]  Ueli Maurer,et al.  A universal statistical test for random bit generators , 1990, Journal of Cryptology.

[40]  Vincent Rijmen,et al.  The Design of Rijndael , 2002, Information Security and Cryptography.

[41]  Pierre L'Ecuyer,et al.  A random number generator based on the combination of four LCGs , 1997 .

[42]  Barrie D. Spurr,et al.  Approximations to the Distribution Function of the Anderson—Darling Test Statistic , 1988 .

[43]  Scott Kirkpatrick,et al.  A very fast shift-register sequence random number generatorjournal of computational physics , 1981 .

[44]  Peter Hellekalek.  Inversive pseudorandom number generators: concepts, results and links , 1995, WSC '95.

[45]  Alan M. Ferrenberg,et al.  Monte Carlo simulations: Hidden errors from "good" random number generators. , 1992, Physical review letters.

[46]  Pierre L'Ecuyer,et al.  Fast combined multiple recursive generators with multipliers of the form a = ±2q ±2r , 2000, WSC '00.

[47]  Richard P. Brent,et al.  Note on Marsaglia's Xorshift Random Number Generators , 2004 .

[48]  G. Marsaglia,et al.  Monkey tests for random number generators , 1993 .

[49]  Pei-Chi Wu,et al.  Multiplicative, congruential random-number generators with multiplier ± 2k1 ± 2k2 and modulus 2p - 1 , 1997, TOMS.

[50]  Pierre L'Ecuyer,et al.  Testing random number generators , 1992, WSC '92.

[51]  P. A. P. Moran,et al.  An introduction to probability theory , 1968 .

[52]  Makoto Matsumoto,et al.  Twisted GFSR generators , 1992, TOMC.

[53]  S. Tezuka.  Uniform Random Numbers: Theory and Practice , 1995 .

[54]  P. L’Ecuyer,et al.  On the lattice structure of certain linear congruential sequences related to AWC/SWB generators , 1994 .

[55]  Pierre L'Ecuyer,et al.  Efficient and portable combined random number generators , 1988, CACM.

[56]  V. S. Stognienko,et al.  A new test for randomness and its application to some cryptographic problems , 2004 .

[57]  Stefan Wegenkittl,et al.  A generalized Φ-divergence for asymptotically multivariate normal models , 2002 .

[58]  Shu Tezuka,et al.  On the lattice structure of the add-with-carry and subtract-with-borrow random number generators , 1993, TOMC.

[59]  Ora E. Percus,et al.  Theory and application of Marsaglia's monkey test for pseudorandom number generators , 1995, TOMC.

[60]  Harald Niederreiter,et al.  Random number generation and Quasi-Monte Carlo methods , 1992, CBMS-NSF regional conference series in applied mathematics.

[61]  Donald E. Knuth,et al.  The art of computer programming. Vol.2: Seminumerical algorithms , 1981 .

[62]  Peter Hellekalek.  Don't trust parallel Monte Carlo! , 1998, Workshop on Parallel and Distributed Simulation.

[63]  Pierre L'Ecuyer,et al.  Random numbers for simulation , 1990, CACM.

[64]  Oded Goldreich,et al.  Modern Cryptography, Probabilistic Proofs and Pseudorandomness , 1998, Algorithms and Combinatorics.

[65]  Elwyn R. Berlekamp,et al.  Algebraic coding theory , 1984, McGraw-Hill series in systems science.

[66]  James L. Massey,et al.  Shift-register synthesis and BCH decoding , 1969, IEEE Trans. Inf. Theory.

[67]  Brian D. Ripley,et al.  Thoughts on pseudorandom number generators , 1990 .

[68]  David Thomas,et al.  The Art in Computer Programming , 2001 .

[69]  Whitlock,et al.  Pseudorandom number generator for massively parallel molecular-dynamics simulations. , 1994, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[70]  Lih-Yuan Deng,et al.  Efficient and portable multiple recursive generators of large order , 2005, TOMC.

[71]  Pierre L'Ecuyer,et al.  On the performance of birthday spacings tests with certain families of random number generators , 2001 .

[72]  Antónia Földes,et al.  The limit distribution of the length of the longest head-run , 1979 .

[73]  Pierre L'Ecuyer,et al.  On the xorshift random number generators , 2005, TOMC.

[74]  Peter Hellekalek,et al.  Empirical evidence concerning AES , 2003, TOMC.

[75]  Harald Niederreiter,et al.  The Linear Complexity Profile and the Jump Complexity of Keystream Sequences , 1991, EUROCRYPT.

[76]  WuPei-Chi.  Multiplicative, congruential random-number generators with multiplier 2k1 2k2 and modulus 2p - 1 , 1997 .

[77]  Pierre L'Ecuyer,et al.  Chapter 3 Uniform Random Number Generation , 2006, Simulation.

[78]  Stefan Wegenkittl,et al.  Entropy estimators and serial tests for ergodic chains , 2001, IEEE Trans. Inf. Theory.

[79]  Pierre L 'ecuyer,et al.  Tests Based on Sum-functions of Spacings for Uniform Random Numbers , 1997 .

[80]  Dirk Fox,et al.  Advanced Encryption Standard (AES) , 1999, Datenschutz und Datensicherheit.

[81]  Pierre L'Ecuyer,et al.  Beware of linear congruential generators with multipliers of the form a = ±2q ±2r , 1999, TOMS.

[82]  Elaine B. Barker,et al.  A Statistical Test Suite for Random and Pseudorandom Number Generators for Cryptographic Applications , 2000 .

[83]  William N. Venables,et al.  Modern Applied Statistics with S-Plus. , 1996 .

[84]  Takashi Kato,et al.  On a nonlinear congruential pseudorandom number generator , 1996, Math. Comput..

[85]  Timothy R. C. Read,et al.  Goodness-Of-Fit Statistics for Discrete Multivariate Data , 1988 .

[86]  Pierre L'Ecuyer,et al.  Distribution properties of multiply-with-c arry random number generators , 1997, Math. Comput..

[87]  Pierre L'Ecuyer,et al.  Tables of maximally equidistributed combined LFSR generators , 1999, Math. Comput..

[88]  Pierre L'Ecuyer,et al.  Good Parameters and Implementations for Combined Multiple Recursive Random Number Generators , 1999, Oper. Res..

[89]  J. Durbin.  Distribution theory for tests based on the sample distribution function , 1973 .

[90]  Pierre L'Ecuyer,et al.  Combined generators with components from different families , 2003, Math. Comput. Simul..

[91]  Pierre L'Ecuyer,et al.  A search for good multiple recursive random number generators , 1993, TOMC.

[92]  Pierre L'Ecuyer,et al.  Uniform random number generation , 1994, Ann. Oper. Res..

[93]  Michael Mascagni,et al.  Algorithm 806: SPRNG: a scalable library for pseudorandom number generation , 1999, TOMS.

[94]  P. Bickel,et al.  Sums of Functions of Nearest Neighbor Distances, Moment Bounds, Limit Theorems and a Goodness of Fit Test , 1983 .

[95]  George Marsaglia,et al.  In: Applications of Number Theory to Numerical Analysis , 1972 .

[96]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[97]  V. A. Monarev,et al.  A New Type of Attacks on Block Ciphers , 2005, Probl. Inf. Transm..

[98]  Glyn Carter,et al.  Aspects of local linear complexity , 1989 .

[99]  J. R. Koehler,et al.  Modern Applied Statistics with S-Plus. , 1996 .

[100]  M. Stephens.  Use of the Kolmogorov-Smirnov, Cramer-Von Mises and Related Statistics without Extensive Tables , 1970 .

[101]  G. Marsaglia,et al.  Some Difficult-to-pass Tests of Randomness , 2022 .

[102]  Ronald L. Wasserstein,et al.  Monte Carlo: Concepts, Algorithms, and Applications , 1997 .

[103]  Pierre L'Ecuyer,et al.  Combined Multiple Recursive Random Number Generators , 1995, Oper. Res..

[104]  G. S. Fishman.  Multiplicative congruential random number generators with modulus 2^{}: an exhaustive analysis for =32 and a partial analysis for =48 , 1990 .

[105]  C. Geiss,et al.  An introduction to probability theory , 2008 .

[106]  Helmut Prodinger,et al.  Digital Search Trees Again Revisited: The Internal Path Length Perspective , 1994, SIAM J. Comput..

[107]  Geoffrey S. Watson,et al.  Distribution Theory for Tests Based on the Sample Distribution Function , 1973 .

[108]  G. Marsaglia,et al.  Matrices and the structure of random number sequences , 1985 .

[109]  Michael S. Waterman,et al.  An extreme value theory for long head runs , 1986 .

[110]  Makoto Matsumoto,et al.  Twisted GFSR generators II , 1994, TOMC.

[111]  P. L'Ecuyer,et al.  Fast combined multiple recursive generators with multipliers of the form a=/spl plusmn/2/sup q//spl plusmn/2/sup r/ , 2000, 2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165).

[112]  Maurice G. Kendall,et al.  Second Paper on Random Sampling Numbers , 1939 .

[113]  Robert M. Ziff,et al.  Four-tap shift-register-sequence random-number generators , 1998 .

[114]  A. L. Ruhkin.  Testing Randomness: A Suite of Statistical Procedures , 2001 .

[115]  M. Luescher,et al.  A Portable High-quality Random Number Generator for Lattice Field Theory Simulations , 1993 .