A polynomial restriction lemma with applications

A polynomial threshold function (PTF) of degree d is a boolean function of the form f=sgn(p), where p is a degree-d polynomial, and sgn is the sign function. The main result of the paper is an almost optimal bound on the probability that a random restriction of a PTF is not close to a constant function, where a boolean function g is called δ-close to constant if, for some vε{1,-1}, we have g(x)=v for all but at most δ fraction of inputs. We show for every PTF f of degree d≥ 1, and parameters 0<δ, r≤ 1/16, that Pr∾ Rr [fρ is not δ-close to constant] ≤ √ #183;(logr-1· logδ-1)O(d2), where ρ ∾ Rr is a random restriction leaving each variable, independently, free with probability r, and otherwise assigning it 1 or -1 uniformly at random. In fact, we show a more general result for random block restrictions: given an arbitrary partitioning of input variables into m blocks, a random block restriction picks a uniformly random block ℓΕ [m] and assigns 1 or -1, uniformly at random, to all variable outside the chosen block ℓ. We prove the Block Restriction Lemma saying that a PTF f of degree d becomes δ-close to constant when hit with a random block restriction, except with probability at most m-1/2 #183; (logm#183; logδ-1)O(d2). As an application of our Restriction Lemma, we prove lower bounds against constant-depth circuits with PTF gates of any degree 1≤ d≪ √logn/loglogn, generalizing the recent bounds against constant-depth circuits with linear threshold gates (LTF gates) proved by Kane and Williams (STOC, 2016) and Chen, Santhanam, and Srinivasan (CCC, 2016). In particular, we show that there is an n-variate boolean function Fn Ε P such that every depth-2 circuit with PTF gates of degree d≥ 1 that computes Fn must have at least (n3/2+1/d)#183; (logn)-O(d2) wires. For constant depths greater than 2, we also show average-case lower bounds for such circuits with super-linear number of wires. These are the first super-linear bounds on the number of wires for circuits with PTF gates. We also give short proofs of the optimal-exponent average sensitivity bound for degree-d PTFs due to Kane (Computational Complexity, 2014), and the Littlewood-Offord type anticoncentration bound for degree-d multilinear polynomials due to Meka, Nguyen, and Vu (Theory of Computing, 2016). Finally, we give derandomized versions of our Block Restriction Lemma and Littlewood-Offord type anticoncentration bounds, using a pseudorandom generator for PTFs due to Meka and Zuckerman (SICOMP, 2013).

[1]  Rocco A. Servedio,et al.  An Average-Case Depth Hierarchy Theorem for Boolean Circuits , 2015, 2015 IEEE 56th Annual Symposium on Foundations of Computer Science.

[2]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[3]  Johan Hå stad The Shrinkage Exponent of de Morgan Formulas is 2 , 1998 .

[4]  Eric Allender,et al.  A note on the power of threshold circuits , 1989, 30th Annual Symposium on Foundations of Computer Science.

[5]  M. Saks Slicing the hypercube , 1993 .

[6]  Michael Sipser,et al.  Parity, circuits, and the polynomial-time hierarchy , 1981, 22nd Annual Symposium on Foundations of Computer Science (sfcs 1981).

[7]  Michael E. Saks,et al.  Size-depth trade-offs for threshold circuits , 1993, SIAM J. Comput..

[8]  Andrew Chi-Chih Yao,et al.  Separating the Polynomial-Time Hierarchy by Oracles (Preliminary Version) , 1985, FOCS.

[9]  Ryan O'Donnell,et al.  Extremal properties of polynomial threshold functions , 2008, J. Comput. Syst. Sci..

[10]  Nathan Linial,et al.  Spectral properties of threshold functions , 1994, Comb..

[11]  Suguru Tamaki,et al.  A Satisfiability Algorithm for Depth Two Circuits with a Sub-Quadratic Number of Symmetric and Threshold Gates , 2016, Electron. Colloquium Comput. Complex..

[12]  Rocco A. Servedio,et al.  Deterministic Approximate Counting for Juntas of Degree-2 Polynomial Threshold Functions , 2013, 2014 IEEE 29th Conference on Computational Complexity (CCC).

[13]  Raghu Meka,et al.  Anti-concentration for Polynomials of Independent Random Variables , 2016, Theory Comput..

[14]  Y. Peres Noise Stability of Weighted Majority , 2004, math/0412377.

[15]  Jehoshua Bruck,et al.  On the Power of Threshold Circuits with Small Weights , 1991, SIAM J. Discret. Math..

[16]  Rocco A. Servedio,et al.  Bounded Independence Fools Halfspaces , 2009, 2009 50th Annual IEEE Symposium on Foundations of Computer Science.

[17]  Johan Håstad,et al.  Almost optimal lower bounds for small depth circuits , 1986, STOC '86.

[18]  Robert O. Winder,et al.  Single stage threshold logic , 1961, SWCT.

[19]  T. Sanders,et al.  Analysis of Boolean Functions , 2012, ArXiv.

[20]  Noam Nisan,et al.  Constant depth circuits, Fourier transform, and learnability , 1989, 30th Annual Symposium on Foundations of Computer Science.

[21]  Ryan Williams Nonuniform ACC Circuit Lower Bounds , 2014, JACM.

[22]  Aravind Srinivasan,et al.  Chernoff-Hoeffding bounds for applications with limited independence , 1995, SODA '93.

[23]  Rahul Santhanam,et al.  Average-Case Lower Bounds and Satisfiability Algorithms for Small Threshold Circuits , 2016, Computational Complexity Conference.

[24]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[25]  Pavel Pudlák,et al.  Threshold circuits of bounded depth , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[26]  J. Littlewood,et al.  On the Number of Real Roots of a Random Algebraic Equation , 1938 .

[27]  A. Carbery,et al.  Distributional and L-q norm inequalities for polynomials over convex bodies in R-n , 2001 .

[28]  M. Anthony Discrete Mathematics of Neural Networks: Selected Topics , 1987 .

[29]  Johan Håstad The Shrinkage Exponent of de Morgan Formulas is 2 , 1998, SIAM J. Comput..

[30]  K. Siu,et al.  Theoretical Advances in Neural Computation and Learning , 1994, Springer US.

[31]  N. Nisan The communication complexity of threshold gates , 1993 .

[32]  Eric Allender,et al.  Amplifying Lower Bounds by Means of Self-Reducibility , 2008, 2008 23rd Annual IEEE Conference on Computational Complexity.

[33]  Rocco A. Servedio,et al.  Hardness results for agnostically learning low-degree polynomial threshold functions , 2011, SODA '11.

[34]  Jehoshua Bruck,et al.  Harmonic Analysis of Polynomial Threshold Functions , 1990, SIAM J. Discret. Math..

[35]  A. Razborov Lower bounds on the size of bounded depth circuits over a complete basis with logical addition , 1987 .

[36]  Moni Naor,et al.  Number-theoretic constructions of efficient pseudo-random functions , 2004, JACM.

[37]  Alexander A. Razborov,et al.  Natural Proofs , 2007 .

[38]  Daniel M. Kane,et al.  Bounded Independence Fools Degree-2 Threshold Functions , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[39]  William H. Kautz,et al.  On the Size of Weights Required for Linear-Input Switching Functions , 1961, IRE Transactions on Electronic Computers.

[40]  Noam Nisan,et al.  The Effect of Random Restrictions on Formula Size , 1993, Random Struct. Algorithms.

[41]  Miklós Ajtai,et al.  ∑11-Formulae on finite structures , 1983, Ann. Pure Appl. Log..

[42]  Timothy M. Chan,et al.  Polynomial Representations of Threshold Functions and Algorithmic Applications , 2016, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[43]  Jehoshua Bruck,et al.  Polynomial threshold functions, AC functions and spectrum norms , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[44]  Daniel M. Kane,et al.  Super-linear gate and super-quadratic wire lower bounds for depth-two and depth-three threshold circuits , 2015, STOC.

[45]  Michael E. Saks,et al.  Approximating Threshold Circuits by Rational Functions , 1994, Inf. Comput..

[46]  Roman Smolensky,et al.  Algebraic methods in the theory of lower bounds for Boolean circuit complexity , 1987, STOC.

[47]  Ryan Williams,et al.  Improving exhaustive search implies superpolynomial lower bounds , 2010, STOC '10.

[48]  Elchanan Mossel,et al.  Noise stability of functions with low influences: Invariance and optimality , 2005, IEEE Annual Symposium on Foundations of Computer Science.

[49]  Thomas Kailath,et al.  Rational approximation techniques for analysis of neural networks , 1994, IEEE Trans. Inf. Theory.

[50]  P. Erdös On a lemma of Littlewood and Offord , 1945 .

[51]  C. K. Chow,et al.  On the characterization of threshold functions , 1961, SWCT.

[52]  Rocco A. Servedio,et al.  A Regularity Lemma and Low-Weight Approximators for Low-Degree Polynomial Threshold Functions , 2014, Theory Comput..

[53]  Rocco A. Servedio,et al.  A Regularity Lemma, and Low-Weight Approximators, for Low-Degree Polynomial Threshold Functions , 2009, 2010 IEEE 25th Annual Conference on Computational Complexity.

[54]  Daniel M. Kane,et al.  A Structure Theorem for Poorly Anticoncentrated Gaussian Chaoses and Applications to the Study of Polynomial Threshold Functions , 2012, 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science.

[55]  Alexander A. Razborov,et al.  Majority gates vs. general weighted threshold gates , 1992, [1992] Proceedings of the Seventh Annual Structure in Complexity Theory Conference.

[56]  V. M. Khrapchenko Method of determining lower bounds for the complexity of P-schemes , 1971 .

[57]  Rocco A. Servedio,et al.  Efficient deterministic approximate counting for low-degree polynomial threshold functions , 2013, STOC.

[58]  Daniel M. Kane A Small PRG for Polynomial Threshold Functions of Gaussians , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[59]  David Zuckerman,et al.  Mining Circuit Lower Bound Proofs for Meta-Algorithms , 2014, computational complexity.

[60]  Ryan O'Donnell,et al.  Noise stability of functions with low influences: Invariance and optimality , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[61]  Prasad Raghavendra,et al.  Average Sensitivity and Noise Sensitivity of Polynomial Threshold Functions , 2014, SIAM J. Comput..

[62]  Daniel M. Kane,et al.  The correct exponent for the Gotsman–Linial Conjecture , 2012, 2013 IEEE Conference on Computational Complexity.

[63]  A. Bonami Étude des coefficients de Fourier des fonctions de $L^p(G)$ , 1970 .

[64]  Andrew Chi-Chih Yao,et al.  ON ACC and threshold circuits , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[65]  Johan Håstad,et al.  An Average-Case Depth Hierarchy Theorem for Higher Depth , 2016, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[66]  Daniel M. Kane A Polylogarithmic PRG for Degree $2$ Threshold Functions in the Gaussian Setting , 2015, Computational Complexity Conference.

[67]  Salil P. Vadhan,et al.  Pseudorandomness , 2012, Found. Trends Theor. Comput. Sci..

[68]  J. Littlewood,et al.  On the number of real roots of a random algebraic equation. II , 1939 .

[69]  Alexander A. Razborov,et al.  On Small Depth Threshold Circuits , 1992, SWAT.

[70]  Rocco A. Servedio,et al.  Attribute-Efficient Learning and Weight-Degree Tradeoffs for Polynomial Threshold Functions , 2012, COLT.

[71]  Avishay Tal,et al.  Shrinkage of De Morgan Formulae by Spectral Techniques , 2014, 2014 IEEE 55th Annual Symposium on Foundations of Computer Science.

[72]  Uri Zwick,et al.  Shrinkage of de Morgan formulae under restriction , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[73]  David Zuckerman,et al.  Pseudorandom generators for polynomial threshold functions , 2009, STOC '10.