Probabilistic Divide-and-Conquer: A New Exact Simulation Method, With Integer Partitions as an Example

We propose a new method, probabilistic divide-and-conquer, for improving the success probability in rejection sampling. For the example of integer partitions, there is an ideal recursive scheme which improves the rejection cost from asymptotically order $n^{3/4}$ to a constant. We show other examples for which a non--recursive, one--time application of probabilistic divide-and-conquer removes a substantial fraction of the rejection sampling cost. We also present a variation of probabilistic divide-and-conquer for generating i.i.d. samples that exploits features of the coupon collector's problem, in order to obtain a cost that is sublinear in the number of samples.

[1]  Hans Rademacher.  A Convergent Series for the Partition Function p(n). , 1937, Proceedings of the National Academy of Sciences of the United States of America.

[2]  Stephen Desalvo,et al.  Probabilistic divide-and-conquer: Deterministic second half , 2014, Adv. Appl. Math..

[3]  Fredrik Johansson,et al.  Efficient implementation of the Hardy-Ramanujan-Rademacher formula , 2012, 1205.5991.

[4]  S. Parsell,et al.  A Hardy–Ramanujan–Rademacher-type formula for (r,s)-regular partitions , 2012, 1901.05327.

[5]  R. Arratia.  On the Amount of Dependence in the Prime Factorization of a Uniform Random Integer , 2013, 1305.0941.

[6]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[7]  M. Waterman,et al.  The Erdos-Renyi Law in Distribution, for Coin Tossing and Sequence Matching , 1990 .

[8]  Boris Pittel,et al.  On a Likely Shape of the Random Ferrers Diagram , 1997 .

[9]  Whitfield Diffie,et al.  Special Feature Exhaustive Cryptanalysis of the NBS Data Encryption Standard , 1977, Computer.

[10]  A. Sills.  Rademacher-type formulas for restricted partition and overpartition functions , 2010 .

[11]  Guy Louchard,et al.  Boltzmann Samplers for the Random Generation of Combinatorial Structures , 2004, Combinatorics, Probability and Computing.

[12]  Thomas M. Cover,et al.  Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing) , 2006 .

[13]  G. Andrews.  The Theory of Partitions: Frontmatter , 1976 .

[14]  P. A. P. Moran,et al.  An introduction to probability theory , 1968 .

[15]  P. Diaconis,et al.  A probabilistic interpretation of the Macdonald polynomials , 2010, 1007.4779.

[16]  Herbert S. Wilf,et al.  A Method and Two Algorithms on the Theory of Partitions , 1975, J. Comb. Theory, Ser. A.

[17]  J. Nicolas.  Sur les entiers $N$ pour lesquels il y a beaucoup de groupes abéliens d’ordre $N$ , 1977 .

[18]  Alain Denise,et al.  Uniform Random Generation of Decomposable Structures Using Floating-Point Arithmetic , 1999, Theor. Comput. Sci..

[19]  A. Nijenhuis.  Combinatorial algorithms , 1975 .

[20]  Laurent Alonso,et al.  Uniform Generation of a Motzkin Word , 1994, Theor. Comput. Sci..

[21]  Michael S. Waterman,et al.  Critical Phenomena in Sequence Matching , 1985 .

[22]  Jeffrey B. Remmel,et al.  Bijective Proofs of Some Classical Partition Identities , 1982, J. Comb. Theory A.

[23]  L. Devroye.  Non-Uniform Random Variate Generation , 1986 .

[24]  Boris G. Pittel,et al.  Random Set Partitions: Asymptotics of Subset Counts , 1997, J. Comb. Theory, Ser. A.

[25]  Exact asymptotic formulas for the coefficients of nonmodular functions , 1991 .

[26]  Charles Knessl,et al.  Partition asymptotics from recursion equations , 1990 .

[27]  Don H. Johnson,et al.  Gauss and the history of the fast Fourier transform , 1984, IEEE ASSP Magazine.

[28]  Andrew Chi-Chih Yao,et al.  The complexity of nonuniform random number generation , 1976 .

[29]  Shmuel Winograd,et al.  Complexity Of Computations , 1978, ACM Annual Conference.

[30]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[31]  Gerardo Rubino,et al.  Introduction to Rare Event Simulation , 2009, Rare Event Simulation using Monte Carlo Methods.

[32]  George E. Andrews,et al.  A Hardy-Ramanujan formula for restricted partitions , 1991 .

[33]  Igor Pak,et al.  Partition bijections, a survey , 2006 .

[34]  J. Propp,et al.  Exact sampling with coupled Markov chains and applications to statistical mechanics , 1996 .

[35]  Béla Bollobás,et al.  Random Graphs, Second Edition , 2001, Cambridge Studies in Advanced Mathematics.

[36]  D. Newman.  The Double Dixie Cup Problem , 1960 .

[37]  Simon Tavare,et al.  Independent Process Approximations for Random Combinatorial Structures , 1994, 1308.3279.

[38]  On the series for the partition function , 1938 .

[39]  G. Hardy,et al.  Asymptotic formulae in combinatory analysis , 1918 .

[40]  P. Erdös,et al.  The distribution of the number of summands in the partitions of a positive integer , 1941 .

[41]  Igor Pak,et al.  Log-concavity of the partition function , 2013, 1310.7982.

[42]  G. Hardy,et al.  Asymptotic Formulaæ in Combinatory Analysis , 1918 .

[43]  Jennifer Morse,et al.  Affine Insertion and Pieri Rules for the Affine Grassmannian , 2006 .

[44]  K. Upton,et al.  A modern approach , 1995 .

[45]  S. Desalvo.  PROBABILISTIC DIVIDE-AND-CONQUER { A NEW METHOD FOR EXACT SIMULATION { AND LOWER BOUND EXPANSIONS FOR RANDOM BERNOULLI MATRICES VIA NOVEL INTEGER PARTITIONS , 2012 .

[46]  Log-concavity of the overpartition function , 2014, 1412.4603.

[47]  Béla Bollobás,et al.  Random Graphs: Notation , 2001 .

[48]  D. H. Lehmer.  On the remainders and convergence of the series for the partition function , 1939 .

[49]  David Thomas,et al.  The Art in Computer Programming , 2001 .

[50]  Donald E. Knuth,et al.  Generating all combinations and partitions , 2008 .

[51]  Brendan D. McKay,et al.  Uniform Generation of Random Regular Graphs of Moderate Degree , 1990, J. Algorithms.

[52]  Albert Nijenhuis,et al.  Combinatorial Algorithms for Computers and Calculators , 1978 .

[53]  Olivier Bodini,et al.  Random Sampling of Plane Partitions , 2006, Combinatorics, Probability and Computing.

[54]  Bert Fristedt,et al.  The structure of random partitions of large integers , 1993 .