A comparison of CPUs, GPUs, FPGAs, and massively parallel processor arrays for random number generation

The future of high-performance computing is likely to rely on the ability to efficiently exploit huge amounts of parallelism. One way of taking advantage of this parallelism is to formulate problems as "embarrassingly parallel" Monte-Carlo simulations, which allow applications to achieve a linear speedup over multiple computational nodes, without requiring a super-linear increase in inter-node communication. However, such applications are reliant on a cheap supply of high quality random numbers, particularly for the three main maximum entropy distributions: uniform, used as a general source of randomness; Gaussian, for discrete-time simulations; and exponential, for discrete-event simulations. In this paper we look at four different types of platform: conventional multi-core CPUs (Intel Core2); GPUs (NVidia GTX 200); FPGAs (Xilinx Virtex-5); and Massively Parallel Processor Arrays (Ambric AM2000). For each platform we determine the most appropriate algorithm for generating each type of number, then calculate the peak generation rate and estimated power efficiency for each device.

[1]  Wayne Luk,et al.  Ziggurat-based hardware Gaussian random number generator , 2005, International Conference on Field Programmable Logic and Applications, 2005..

[2]  Wayne Luk,et al.  A hardware Gaussian noise generator using the Box-Muller method and its error analysis , 2006, IEEE Transactions on Computers.

[3]  Wayne Luk,et al.  Non-Uniform Random Number Generation Through Piecewise Linear Approximations , 2006, 2006 International Conference on Field Programmable Logic and Applications.

[4]  G. Marsaglia,et al.  The Ziggurat Method for Generating Random Variables , 2000 .

[5]  Pierre L'Ecuyer,et al.  Tables of maximally equidistributed combined LFSR generators , 1999, Math. Comput..

[6]  L. Devroye.  Non-Uniform Random Variate Generation , 1986 .

[7]  Wayne Luk,et al.  A Gaussian noise generator for hardware-based simulations , 2004, IEEE Transactions on Computers.

[8]  Chris S. Wallace.  Fast pseudorandom generators for normal and exponential variates , 1996, TOMS.

[9]  Wayne Luk,et al.  A hardware Gaussian noise generator using the Wallace method , 2005, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[10]  Pierre L'Ecuyer,et al.  Improved long-period generators based on linear recurrences modulo 2 , 2004, TOMS.

[11]  Makoto Matsumoto,et al.  SIMD-Oriented Fast Mersenne Twister: a 128-bit Pseudorandom Number Generator , 2008 .

[12]  Wayne Luk,et al.  High Quality Uniform Random Number Generation Using LUT Optimised State-transition Matrices , 2007, J. VLSI Signal Process..

[13]  Wayne Luk,et al.  Hardware Generation of Arbitrary Random Number Distributions From Uniform Distributions Via the Inversion Method , 2007, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[14]  Hiroaki Kitano,et al.  FPGA Implementation of a Data-Driven Stochastic Biochemical Simulator with the Next Reaction Method , 2007, 2007 International Conference on Field Programmable Logic and Applications.

[15]  Wayne Luk,et al.  Sampling from the exponential distribution using independent Bernoulli variates , 2008, 2008 International Conference on Field Programmable Logic and Applications.

[16]  G. Forsythe.  Von Neumann''s comparison method for random sampling from the normal and other distributions. , 1972 .

[17]  M. E. Muller,et al.  A Note on the Generation of Random Normal Deviates , 1958 .

[18]  R. Brent.  Algorithm 488: A Gaussian pseudo-random number generator , 1974, CACM.

[19]  Wayne Luk,et al.  Credit Risk Modelling using Hardware Accelerated Monte-Carlo Simulation , 2008, 2008 16th International Symposium on Field-Programmable Custom Computing Machines.

[20]  Rainer A. Rueppel,et al.  Correlation Immunity and the Summation Generator , 1985, CRYPTO.

[21]  Wayne Luk,et al.  Gaussian random number generators , 2007, CSUR.

[22]  Joachim H. Ahrens,et al.  Computer methods for sampling from the exponential and normal distributions , 1972, CACM.