Separating AC0 from depth-2 majority circuits

We prove that AC<sup>0</sup> cannot be efficiently simulated by MAJºMAJ circuits. Namely, we construct an AC<sup>0</sup> circuit of depth 3 that requires MAJºMAJ circuits of size 2<sup>Ω(n<sup>1/5</sup>)</sup>. This matches Allender's classic result that AC<sup>0</sup> can be simulated by MAJºMAJºMAJ circuits of quasipolynomial size. Our proof is based on communication complexity. To obtain the above result, we develop a novel technique for communication lower bounds, the Degree/Discrepancy Theorem. This technique is a separate contribution of our paper. It translates lower bounds on the threshold degree of a Boolean function into upper bounds on the discrepancy of a related function. Upper bounds on the discrepancy, in turn, immediately imply communication lower bounds as well as lower bounds against threshold circuits. As part of our proof, we use the Degree/Discrepancy Theorem to obtain an explicit AC<sup>0</sup> circuit of depth 3 that has discrepancy 2<sup>-Ω(n<sup>1/5</sup>)</sup>, under an explicit distribution. This yields the first known AC<sup>0</sup> function with exponentially small discrepancy. Finally, we apply our work to learning theory, showing that polynomial-size DNF and CNF formulas have margin complexity 2<sup>Ω(n<sup>1/5</sup>)</sup>.

[1]  Nathan Linial,et al.  Lower bounds in communication complexity based on factorization norms , 2007, STOC '07.

[2]  Alexander A. Sherstov The pattern matrix method for lower bounds on quantum communication , 2008, STOC '08.

[3]  Rocco A. Servedio,et al.  Learning intersections of halfspaces with a margin , 2004, J. Comput. Syst. Sci..

[4]  Jehoshua Bruck,et al.  Depth efficient neural networks for division and related problems , 1993, IEEE Trans. Inf. Theory.

[5]  Alexander A. Sherstov Halfspace Matrices , 2007, Twenty-Second Annual IEEE Conference on Computational Complexity (CCC'07).

[6]  Ran Raz,et al.  The BNS-Chung criterion for multi-party communication complexity , 2000, computational complexity.

[7]  A. Razborov Quantum communication complexity of symmetric predicates , 2002, quant-ph/0204025.

[8]  Marvin Minsky,et al.  Perceptrons: expanded edition , 1988 .

[9]  Rocco A. Servedio,et al.  Learning DNF in time , 2001, STOC '01.

[10]  Ryan O'Donnell,et al.  New degree bounds for polynomial threshold functions , 2003, STOC '03.

[11]  Emanuele Viola,et al.  Improved Separations between Nondeterministic and Randomized Multiparty Communication , 2008, APPROX-RANDOM.

[12]  Nikolai K. Vereshchagin,et al.  On Computation and Communication with Small Bias , 2007, Computational Complexity Conference.

[13]  Satyanarayana V. Lokam,et al.  Relations Between Communication Complexity, Linear Arrangements, and Computational Complexity , 2001, FSTTCS.

[14]  Kai-Yeung Siu,et al.  On Optimal Depth Threshold Circuits for Multiplication and Related Problems , 1994, SIAM J. Discret. Math..

[15]  David K. Smith Theory of Linear and Integer Programming , 1987 .

[16]  Alexander A. Sherstov SeparatingAC0 from Depth-2 Majority Circuits , 2009, SIAM J. Comput..

[17]  James Aspnes,et al.  The expressive power of voting polynomials , 1994, Comb..

[18]  Jürgen Forster A linear lower bound on the unbounded error probabilistic communication complexity , 2002, J. Comput. Syst. Sci..

[19]  Eric Allender,et al.  A note on the power of threshold circuits , 1989, 30th Annual Symposium on Foundations of Computer Science.

[20]  Jehoshua Bruck,et al.  On the Power of Threshold Circuits with Small Weights , 1991, SIAM J. Discret. Math..

[21]  Ryan O'Donnell,et al.  Extremal properties of polynomial threshold functions , 2003, 18th IEEE Annual Conference on Computational Complexity, 2003. Proceedings..

[22]  Rocco A. Servedio,et al.  Learning DNF in time 2Õ(n1/3) , 2004, J. Comput. Syst. Sci..

[23]  Pavel Pudlák,et al.  Computing Boolean functions by polynomials and threshold circuits , 1998, computational complexity.

[24]  Pavel Pudlák,et al.  On the Computational Power of Depth-2 Circuits with Threshold and Modulo Gates , 1997, Theor. Comput. Sci..

[25]  Hartmut Klauck Lower Bounds for Quantum Communication Complexity , 2007, SIAM J. Comput..

[26]  Fan Chung Graham,et al.  Communication Complexity and Quasi Randomness , 1993, SIAM J. Discret. Math..

[27]  Arkadev Chattopadhyay,et al.  Multiparty Communication Complexity of Disjointness , 2008, Electron. Colloquium Comput. Complex..

[28]  Pavel Pudlák,et al.  Threshold circuits of bounded depth , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[29]  Alexander A. Sherstov Powering requires threshold depth 3 , 2007, Inf. Process. Lett..

[30]  N. Nisan The communication complexity of threshold gates , 1993 .

[31]  Albert B Novikoff,et al.  ON CONVERGENCE PROOFS FOR PERCEPTRONS , 1963 .

[32]  Alexander A. Sherstov,et al.  Improved Lower Bounds for Learning Intersections of Halfspaces , 2006, COLT.

[33]  Alexander A. Sherstov,et al.  Unconditional lower bounds for learning intersections of halfspaces , 2007, Machine Learning.

[34]  Noam Nisan,et al.  Multiparty Protocols, Pseudorandom Generators for Logspace, and Time-Space Trade-Offs , 1992, J. Comput. Syst. Sci..

[35]  A. Chattopadhyay Discrepancy and the Power of Bottom Fan-in in Depth-three Circuits , 2007, FOCS 2007.

[36]  Nathan Linial,et al.  Learning Complexity vs. Communication Complexity , 2008, Computational Complexity Conference.

[37]  Alexander A. Sherstov,et al.  A Lower Bound for Agnostically Learning Disjunctions , 2007, COLT.

[38]  Toniann Pitassi,et al.  Separating NOF communication complexity classes RP and NP , 2008, Electron. Colloquium Comput. Complex..

[39]  Troy Lee,et al.  Disjointness Is Hard in the Multi-party Number-on-the-Forehead Model , 2008, Computational Complexity Conference.

[40]  Adam R. Klivans,et al.  Learning DNF in time 2 Õ(n 1/3 ) . , 2001, STOC 2001.

[41]  Peter Frankl,et al.  Complexity classes in communication complexity theory , 1986, 27th Annual Symposium on Foundations of Computer Science (sfcs 1986).

[42]  Adi Shraibman,et al.  Lower bounds in communication complexity based on factorization norms , 2009 .

[43]  Alexander A. Razborov,et al.  On the Distributional Complexity of Disjointness , 1992, Theor. Comput. Sci..

[44]  Janos Simon,et al.  Probabilistic Communication Complexity , 1986, J. Comput. Syst. Sci..

[45]  Alexander A. Razborov,et al.  Majority gates vs. general weighted threshold gates , 2005, computational complexity.

[46]  Alexander A. Razborov,et al.  The Sign-Rank of AC^O , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[47]  Alexander A. Sherstov The unbounded-error communication complexity of symmetric functions , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[48]  Alexander A. Sherstov Communication Lower Bounds Using Dual Polynomials , 2008, Bull. EATCS.

[49]  Yaoyun Shi,et al.  Quantum communication complexity of block-composed functions , 2007, Quantum Inf. Comput..

[50]  E. Kushilevitz,et al.  Communication Complexity: Basics , 1996 .

[51]  Ran Raz,et al.  Fourier analysis for probabilistic communication complexity , 1995, computational complexity.

[52]  Marek Karpinski,et al.  Simulating Threshold Circuits by Majority Circuits , 1998, SIAM J. Comput..