The Fast Loaded Dice Roller: A Near-Optimal Exact Sampler for Discrete Probability Distributions

This paper introduces a new algorithm for the fundamental problem of generating a random integer from a discrete probability distribution using a source of independent and unbiased random coin flips. We prove that this algorithm, which we call the Fast Loaded Dice Roller (FLDR), is highly efficient in both space and time: (i) the size of the sampler is guaranteed to be linear in the number of bits needed to encode the input distribution; and (ii) the expected number of bits of entropy it consumes per sample is at most 6 bits more than the information-theoretically optimal rate. We present fast implementations of the linear-time preprocessing and near-optimal sampling algorithms using unsigned integer arithmetic. Empirical evaluations on a broad set of probability distributions establish that FLDR is 2x-10x faster in both preprocessing and sampling than multiple baseline algorithms, including the widely-used alias and interval samplers. It also uses up to 10000x less space than the information-theoretically optimal sampler, at the expense of less than 1.5x runtime overhead.

[1]  Frederik Vercauteren,et al.  High Precision Discrete Gaussian Sampling on FPGAs , 2013, Selected Areas in Cryptography.

[2]  Manuel Blum.  Independent unbiased coin flips from a correlated biased source—A finite state markov chain , 1986, Comb..

[3]  J. L. Harrison,et al.  The Government Printing Office , 1968, American Journal of Pharmaceutical Education.

[4]  L. Devroye.  Non-Uniform Random Variate Generation , 1986 .

[5]  Julia Abrahams,et al.  Generation of discrete distributions from biased coins , 1996, IEEE Trans. Inf. Theory.

[6]  Manuel Blum,et al.  Independent unbiased coin flips from a correlated biased source—A finite state markov chain , 1984, Comb..

[7]  Y. Peres.  Iterating Von Neumann's Procedure for Extracting Random Bits , 1992 .

[8]  Andrew Chi-Chih Yao,et al.  The complexity of nonuniform random number generation , 1976 .

[9]  Luc Devroye,et al.  A note on approximations in random variate generation , 1982 .

[10]  Sergio Verdú,et al.  Approximation theory of output statistics , 1993, IEEE Trans. Inf. Theory.

[11]  P. Elias.  The Efficient Construction of an Unbiased Random Sequence , 1972 .

[12]  Tomohiko Uyematsu,et al.  Two Algorithms for Random Number Generation Implemented by Using Arithmetic of Limited Precision , 2003, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[13]  Martin C. Rinard,et al.  Optimal approximate sampling from discrete probability distributions , 2020, Proc. ACM Program. Lang..

[14]  Lenore Blum,et al.  Complexity and Real Computation , 1997, Springer New York.

[15]  John F. Monahan,et al.  Accuracy in random number generation , 1985 .

[16]  Milena Mihail,et al.  Efficient Generation ε-close to G(n,p) and Generalizations , 2012, ArXiv.

[17]  Thomas Prest,et al.  Gaussian Sampling in Lattice-Based Cryptography , 2015 .

[18]  Jérémie O. Lumbroso.  Optimal Discrete Uniform Generation from Coin Flips, and Applications , 2013, ArXiv.

[19]  J. R. Roche,et al.  Efficient Generation Of Random Variables From Biased Coins , 1991, Proceedings. 1991 IEEE International Symposium on Information Theory.

[20]  Alastair J. Walker,et al.  An Efficient Method for Generating Discrete Random Variables with General Distributions , 1977, TOMS.

[21]  Mamoru Hoshi,et al.  Interval algorithm for random number generation , 1997, IEEE Trans. Inf. Theory.

[22]  Tobias Friedrich,et al.  Exact and Efficient Generation of Geometric Random Variates and Random Graphs , 2013, ICALP.

[23]  Michael D. Vose,et al.  A Linear Algorithm For Generating Random Numbers With a Given Distribution , 1991, IEEE Trans. Software Eng..

[24]  Dexter Kozen,et al.  Coalgebraic Tools for Randomness-Conserving Protocols , 2018, RAMiCS.

[25]  Michael C. Loui,et al.  Randomizing Functions: Simulation of a Discrete Probability Distribution Using a Source of Unknown Distribution , 2006, IEEE Transactions on Information Theory.

[26]  Konstantinos Panagiotou,et al.  Efficient Sampling Methods for Discrete Distributions , 2012, ICALP.

[27]  Luc Devroye,et al.  Sampling with arbitrary precision , 2015, ArXiv.