Time hierarchies for sampling distributions

We show that "a little more time gives a lot more power to sampling algorithms." We prove that for every constant k ≥ 2, every polynomial time bound t, and every polynomially small ε, there exists a family of distributions on k elements that can be sampled exactly in polynomial time but cannot be sampled within statistical distance 1-1/k-ε in time t. This implies the following general time hierarchy for sampling distributions on arbitrary-size domains such as {0,1}n: For every polynomial time bound t and every constant ε>0, there exists a family of distributions that can be sampled exactly in polynomial time but cannot be sampled within statistical distance 1-ε in time t. Our proof involves reducing the problem to a communication problem over a certain type of noisy channel. To solve the latter problem we use a type of list-decodable code for a setting where there is no bound on the number of errors but each error gives more information than an erasure. This type of code can be constructed using certain known traditional list-decodable codes, but we give a new construction that is elementary, self-contained, and tailored to this setting.

[1]  Sampath Kannan,et al.  A Quasi-Polynomial-Time Algorithm for Sampling Words from a Context-Free Language , 1997, Inf. Comput..

[2]  Mahdi Cheraghchi,et al.  Almost-Uniform Sampling of Points on High-Dimensional Algebraic Varieties , 2009, STACS.

[3]  Venkatesan Guruswami,et al.  Linear time encodable and list decodable codes , 2003, STOC '03.

[4]  Amnon Ta-Shma,et al.  Extractor codes , 2004, IEEE Trans. Inf. Theory.

[5]  Alistair Sinclair,et al.  Random walks on truncated cubes and sampling 0-1 knapsack solutions , 1999, 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039).

[6]  Luca Trevisan Average-case Complexity , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[7]  Richard M. Karp,et al.  Monte-Carlo Approximation Algorithms for Enumeration Problems , 1989, J. Algorithms.

[8]  Dmitry Itsykson Structural complexity of AvgBPP , 2010, Ann. Pure Appl. Log..

[9]  Venkatesan Guruswami,et al.  The Existence of Concatenated Codes List-Decodable up to the Hamming Bound , 2010, IEEE Transactions on Information Theory.

[10]  Amnon Ta-Shma,et al.  Extractor codes , 2001, IEEE Transactions on Information Theory.

[11]  Venkatesan Guruswami,et al.  Expander-based constructions of efficiently decodable codes , 2001, Proceedings 2001 IEEE International Conference on Cluster Computing.

[12]  J. Hartmanis,et al.  On the Computational Complexity of Algorithms , 1965 .

[13]  Prasad Tetali,et al.  Simple Markov-chain algorithms for generating bipartite graphs and tournaments , 1997, SODA '97.

[14]  J. Komlos,et al.  On the Size of Separating Systems and Families of Perfect Hash Functions , 1984 .

[15]  Scott Aaronson The Equivalence of Sampling and Searching , 2013, Theory of Computing Systems.

[16]  Boaz Barak,et al.  A Probabilistic-Time Hierarchy Theorem for "Slightly Non-uniform" Algorithms , 2002, RANDOM.

[17]  Emanuele Viola,et al.  Extractors for Circuit Sources , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[18]  David Bruce Wilson,et al.  How to Get a Perfectly Random Sample from a Generic Markov Chain and Generate a Random Spanning Tree of a Directed Graph , 1998, J. Algorithms.

[19]  Venkatesan Guruswami,et al.  Better Binary List Decodable Codes Via Multilevel Concatenation , 2007, IEEE Transactions on Information Theory.

[20]  D. Wilson Mixing times of lozenge tiling and card shuffling Markov chains , 2001, math/0102193.

[21]  Martin E. Dyer,et al.  Faster random generation of linear extensions , 1999, SODA '98.

[22]  Dieter van Melkebeek,et al.  Space Hierarchy Results for Randomized and other Semantic Models , 2010, computational complexity.

[23]  Konstantin Pervyshev On Heuristic Time Hierarchies , 2007, Twenty-Second Annual IEEE Conference on Computational Complexity (CCC'07).

[24]  J. Körner Fredman-Kolmo´s bounds and information theory , 1986 .

[25]  Martin E. Dyer,et al.  Approximately counting integral flows and cell-bounded contingency tables , 2005, STOC '05.

[26]  Anindya De,et al.  Extractors and Lower Bounds for Locally Samplable Sources , 2011, TOCT.

[27]  Dana Randall,et al.  Markov chain algorithms for planar lattice structures , 1995, Proceedings of IEEE 36th Annual Foundations of Computer Science.

[28]  Lance Fortnow,et al.  Hierarchy theorems for probabilistic polynomial time , 2004, 45th Annual IEEE Symposium on Foundations of Computer Science.

[29]  David Bruce Wilson,et al.  Generating random spanning trees more quickly than the cover time , 1996, STOC '96.

[30]  Shachar Lovett,et al.  Bounded-Depth Circuits Cannot Sample Good Codes , 2011, 2011 IEEE 26th Annual Conference on Computational Complexity.

[31]  Michael J. Fischer,et al.  Separating Nondeterministic Time Complexity Classes , 1978, JACM.

[32]  Konstantin Pervyshev Time Hierarchies for Computations with a Bit of Advice , 2005, Electron. Colloquium Comput. Complex..

[33]  Adam Tauman Kalai Generating Random Factored Numbers, Easily , 2002, SODA '02.

[34]  Emanuele Viola,et al.  The Complexity of Distributions , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[35]  Luca Trevisan,et al.  Hierarchies for semantic classes , 2005, STOC '05.

[36]  Venkatesan Guruswami,et al.  Explicit Codes Achieving List Decoding Capacity: Error-Correction With Optimal Redundancy , 2005, IEEE Transactions on Information Theory.

[37]  Eric Vigoda,et al.  A Note on the Glauber Dynamics for Sampling Independent Sets , 2001, Electron. J. Comb..

[38]  M. Luby,et al.  Fast convergence of the Glauber dynamics for sampling independent sets , 1999 .

[39]  Eric Vigoda,et al.  A survey on the use of Markov chains to randomly sample colorings , 2006 .

[40]  Oded Goldreich,et al.  On the Implementation of Huge Random Objects , 2003, SIAM J. Comput..

[41]  Luca Trevisan,et al.  From logarithmic advice to single-bit advice , 2004, Electron. Colloquium Comput. Complex..

[42]  Prasad Tetali,et al.  Simple Markov-Chain Algorithms for Generating Bipartite Graphs and Tournaments (Extended Abstract) , 1999, SODA.

[43]  Dieter van Melkebeek,et al.  A Generic Time Hierarchy with One Bit of Advice , 2007, computational complexity.

[44]  Leslie Ann Goldberg,et al.  Randomly sampling molecules , 1997, SODA '97.

[45]  Stephen A. Cook,et al.  A hierarchy for nondeterministic time complexity , 1972, J. Comput. Syst. Sci..

[46]  Shachar Lovett,et al.  Large Deviation Bounds for Decision Trees and Sampling Lower Bounds for AC0-Circuits , 2012, 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science.

[47]  Peter Elias,et al.  Zero error capacity under list decoding , 1988, IEEE Trans. Inf. Theory.

[48]  Aleksander Madry,et al.  Faster Generation of Random Spanning Trees , 2009, 2009 50th Annual IEEE Symposium on Foundations of Computer Science.

[49]  Stanislav Zák,et al.  A Turing Machine Time Hierarchy , 1983, Theor. Comput. Sci..

[50]  Dima Grigoriev,et al.  Time hierarchies for cryptographic function inversion with advice , 2009, Electron. Colloquium Comput. Complex..

[51]  Eric Bach,et al.  How to Generate Factored Random Numbers , 1988, SIAM J. Comput..

[52]  Russell Martin,et al.  Exact counting of Euler tours for generalized series-parallel graphs , 2012, J. Discrete Algorithms.

[53]  Eric Vigoda,et al.  Fast convergence of the Glauber dynamics for sampling independent sets , 1999, Random Struct. Algorithms.

[54]  Eric Vigoda,et al.  A polynomial-time approximation algorithm for the permanent of a matrix with nonnegative entries , 2004, JACM.

[55]  Mark Jerrum,et al.  Approximating the Permanent , 1989, SIAM J. Comput..

[56]  Dana Randall,et al.  Sampling stable marriages: why spouse-swapping won't work , 2008, SODA '08.