Computational applications of noise sensitivity

This thesis is concerned with the study of the noise sensitivity of boolean functions and its applications in theoretical computer science. Noise sensitivity is defined as follows: Let f be a boolean function and let e ∈ (0, h) be a parameter. Suppose a uniformly random string x is picked, and y is formed by flipping each bit of x independently with probability e. Then the noise sensitivity of f at e is defined to be the probability that f(x) and f (y) differ. In this thesis we investigate the noise sensitivity of various classes of boolean functions, including majorities and recursive majorities, boolean threshold functions, and monotone functions. Following this we give new complexity-theoretic and algorithmic applications of noise sensitivity: (1) Regarding computational hardness amplification, we prove a general direct product theorem that tightly characterizes the hardness of a composite function g ⊗ f in terms of an assumed hardness of f and the noise sensitivity of g. The theorem lets us prove a new result about the hardness on average of NP: If NP is (1 − poly(n))-hard for circuits of polynomial size, then it is in fact (h + o(1))-hard for circuits of polynomial size. (2) In the field of computational learning theory, we show that any class whose functions have low noise sensitivity is efficiently learnable. Using our noise sensitivity estimates for functions of boolean halfspaces we obtain new polynomial and quasipolynomial time algorithms for learning intersections, thresholds, and other functions of halfspaces. From noise sensitivity considerations we also give a polynomial time algorithm for learning polynomial-sized DNFs under the “Random Walk” model; we also give the first algorithm that learns the class of “junta” functions with efficiency better than that of the brute force algorithm. (3) Finally, we introduce a new collective coin-flipping problem whose study is equivalent to the study of “higher moments” of the noise sensitivity problem. We prove several results about this extension, and find optimal or near-optimal choices for the coin-flipping function for all asymptotic limits of the parameters. Our techniques include a novel application of the reverse Bonami-Beckner inequality. (Copies available exclusively from MIT Libraries, Rm. 14-0551, Cambridge, MA 02139-4307. Ph. 617-253-5668; Fax 617-253-1690.)

[1]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .

[2]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[3]  H. D. Block.  The perceptron: a model for brain functioning. I , 1962 .

[4]  D. Kleitman.  Families of Non-disjoint subsets* , 1966 .

[5]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[6]  A. Bonami.  Ensembles $\Lambda (p)$ dans le dual de $D^\infty $ , 1968 .

[7]  W. Beckner.  Inequalities in Fourier analysis , 1975 .

[8]  L. Gross.  LOGARITHMIC SOBOLEV INEQUALITIES. , 1975 .

[9]  Robert J. McEliece,et al.  New upper bounds on the rate of a code via the Delsarte-MacWilliams inequalities , 1977, IEEE Trans. Inf. Theory.

[10]  V. Sazonov.  Normal Approximation: Some Recent Advances , 1981 .

[11]  Andrew Chi-Chih Yao,et al.  Theory and application of trapdoor functions , 1982, 23rd Annual Symposium on Foundations of Computer Science (sfcs 1982).

[12]  C. Borell.  Positivity improving operators and hypercontractivity , 1982 .

[13]  I. Miheev,et al.  Trigonometric series with gaps , 1983 .

[14]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[15]  Don Coppersmith,et al.  Matrix multiplication via arithmetic progressions , 1987, STOC.

[16]  David Haussler,et al.  Occam's Razor , 1987, Inf. Process. Lett..

[17]  Nathan Linial,et al.  The influence of variables on Boolean functions , 1988, [Proceedings 1988] 29th Annual Symposium on Foundations of Computer Science.

[18]  Noam Nisan,et al.  Constant depth circuits, Fourier transform, and learnability , 1989, 30th Annual Symposium on Foundations of Computer Science.

[19]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[20]  Nathan Linial,et al.  Collective Coin Flipping , 1989, Adv. Comput. Res..

[21]  Manfred K. Warmuth,et al.  Learning integer lattices , 1990, COLT '90.

[22]  Karsten A. Verbeurgt.  Learning DNF under the uniform distribution in quasi-polynomial time , 1990, COLT '90.

[23]  Eric B. Baum,et al.  A Polynomial Time Algorithm That Learns Two Hidden Unit Nets , 1990, Neural Computation.

[24]  Michael Kearns,et al.  Computational complexity of machine learning , 1990, ACM distinguished dissertations.

[25]  Yishay Mansour,et al.  Learning monotone ku DNF formulas on product distributions , 1991, COLT '91.

[26]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[27]  Eric B. Baum,et al.  Neural net algorithms that learn in polynomial time from examples and queries , 1991, IEEE Trans. Neural Networks.

[28]  Eyal Kushilevitz,et al.  Learning decision trees using the Fourier spectrum , 1991, STOC '91.

[29]  Noam Nisan,et al.  BPP has subexponential time simulations unless EXPTIME has publishable proofs , 1991, [1991] Proceedings of the Sixth Annual Structure in Complexity Theory Conference.

[30]  Yishay Mansour,et al.  An O(nlog log n) learning algorithm for DNF under the uniform distribution , 1992, COLT '92.

[31]  N. Linial,et al.  The influence of variables in product spaces , 1992 .

[32]  Michael Kharitonov,et al.  Cryptographic lower bounds for learnability of Boolean functions on the uniform distribution , 1992, COLT '92.

[33]  Michael Kearns,et al.  Efficient noise-tolerant learning from statistical queries , 1993, STOC.

[34]  L. Gross.  Logarithmic Sobolev inequalities and contractivity properties of semigroups , 1993 .

[35]  Yishay Mansour,et al.  Weakly learning DNF and characterizing statistical query learning using Fourier analysis , 1994, STOC '94.

[36]  Peter L. Bartlett,et al.  Exploiting random walks for learning , 1994, COLT '94.

[37]  D. Bakry.  L'hypercontractivité et son utilisation en théorie des semigroupes , 1994 .

[38]  Yishay Mansour,et al.  Learning Boolean Functions via the Fourier Transform , 1994 .

[39]  Avrim Blum,et al.  Relevant Examples and Relevant Features: Thoughts from Computational Learning Theory , 1994 .

[40]  M. Talagrand.  Concentration of measure and isoperimetric inequalities in product spaces , 1994, math/9406212.

[41]  M. Talagrand.  On Russo's Approximate Zero-One Law , 1994 .

[42]  Jeffrey C. Jackson,et al.  An efficient membership-query algorithm for learning DNF with respect to the uniform distribution , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[43]  Donna K. Slonim,et al.  Learning with unreliable boundary queries , 1995, COLT '95.

[44]  Mihir Bellare,et al.  Free bits, PCPs and non-approximability-towards tight results , 1995, Proceedings of IEEE 36th Annual Foundations of Computer Science.

[45]  Russell Impagliazzo,et al.  Hard-core distributions for somewhat hard problems , 1995, Proceedings of IEEE 36th Annual Foundations of Computer Science.

[46]  Nathan Linial,et al.  On the distance distribution of codes , 1995, IEEE Trans. Inf. Theory.

[47]  Umesh V. Vazirani,et al.  A Markovian extension of Valiant's learning model , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[48]  Mario Marchand,et al.  On learning ?-perceptron networks on the uniform distribution , 1996, Neural Networks.

[49]  Michel Talagrand,et al.  How much are increasing sets positively correlated? , 1996, Comb..

[50]  G. Kalai,et al.  Every monotone graph property has a sharp threshold , 1996 .

[51]  Nader H. Bshouty,et al.  On the Fourier spectrum of monotone functions , 1996, JACM.

[52]  M. An.  Log-Concave Probability Distributions: Theory and Statistical Testing , 1996 .

[53]  Joachim von zur Gathen,et al.  Polynomials with two values , 1997, Comb..

[54]  Santosh S. Vempala,et al.  A random sampling based algorithm for learning the intersection of half-spaces , 1997, Proceedings 38th Annual Symposium on Foundations of Computer Science.

[55]  Avi Wigderson,et al.  P = BPP if E requires exponential circuits: derandomizing the XOR lemma , 1997, STOC '97.

[56]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..

[57]  Avrim Blum,et al.  Learning an Intersection of a Constant Number of Halfspaces over a Uniform Distribution , 1997, J. Comput. Syst. Sci..

[58]  Michel Talagrand,et al.  On boundaries and influences , 1997, Comb..

[59]  J. Bourgain,et al.  Influences of Variables and Threshold Intervals under Group Symmetries , 1997 .

[60]  Ueli Maurer,et al.  Information-Theoretically Secure Secret-Key Agreement by NOT Authenticated Public Discussion , 1997, EUROCRYPT.

[61]  Mihir Bellare,et al.  Free Bits, PCPs, and Nonapproximability-Towards Tight Results , 1998, SIAM J. Comput..

[62]  Ehud Friedgut,et al.  Boolean Functions With Low Average Sensitivity Depend On Few Coordinates , 1998, Comb..

[63]  Stephen Kwek,et al.  PAC Learning Intersections of Halfspaces with Membership Queries , 1998, Algorithmica.

[64]  I. Benjamini,et al.  Noise sensitivity of Boolean functions and applications to percolation , 1998, math/9811157.

[65]  J. Kahn,et al.  On the number of copies of one hypergraph in another , 1998 .

[66]  Luca Trevisan,et al.  Pseudorandom generators without the XOR lemma , 1999, Proceedings. Fourteenth Annual IEEE Conference on Computational Complexity (Formerly: Structure in Complexity Theory Conference) (Cat.No.99CB36317).

[67]  David Gamarnik,et al.  Extension of the PAC framework to finite and countable Markov chains , 1999, COLT '99.

[68]  Nader H. Bshouty,et al.  More efficient PAC-learning of DNF with membership queries under the uniform distribution , 1999, COLT '99.

[69]  Leonid A. Levin,et al.  A Pseudorandom Generator from any One-way Function , 1999, SIAM J. Comput..

[70]  Rocco A. Servedio,et al.  Boosting and hard-core sets , 1999, 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039).

[71]  Nader H. Bshouty,et al.  Uniform-distribution attribute noise learnability , 1999, COLT '99.

[72]  Tatsuie Tsukiji,et al.  Finding Relevant Variables in PAC Model with Membership Queries , 1999, ALT.

[73]  Oded Schramm,et al.  Trees, Not Cubes: Hypercontractivity, Cosiness, and Noise Stability , 1999, math/9902116.

[74]  E. Friedgut,et al.  Sharp thresholds of graph properties, and the -sat problem , 1999 .

[75]  S. Bobkov,et al.  Discrete isoperimetric and Poincaré-type inequalities , 1999 .

[76]  Noise sensitivity on continuous products: an answer to an old question of J. Feldman , 1999, math/9907011.

[77]  V. Statulevičius,et al.  Limit Theorems of Probability Theory , 2000 .

[78]  Impossibility of Black-Box Reduction from Non-Adaptively to Adaptively Secure Coin-Flipping , 2000, Electron. Colloquium Comput. Complex..

[79]  Daniel Stefankovic.  Fourier Transforms in Computer Science , 2000 .

[80]  Rocco A. Servedio,et al.  Efficient algorithms in computational learning theory , 2001 .

[81]  Anna Bernasconi.  On a hierarchy of Boolean functions hard to compute in constant depth , 2001, Discret. Math. Theor. Comput. Sci..

[82]  Johan Håstad,et al.  Some optimal inapproximability results , 2001, JACM.

[83]  Dana Ron,et al.  Proclaiming Dictators and Juntas or Testing Boolean Formulae , 2001, RANDOM-APPROX.

[84]  Vitaly Feldman,et al.  On Using Extended Statistical Queries to Avoid Membership Queries , 2001, J. Mach. Learn. Res..

[85]  Assaf Naor,et al.  Boolean functions whose Fourier transform is concentrated on the first two levels , 2002, Adv. Appl. Math..

[86]  S. Bezrukov.  Isoperimetric Problems in Discrete Spaces , 2002 .

[87]  Michael O. Rabin,et al.  Hyper-Encryption and Everlasting Security , 2002, STACS.

[88]  Rocco A. Servedio,et al.  Learnability beyond AC0 , 2002, STOC '02.

[89]  Irit Dinur,et al.  The importance of being biased , 2002, STOC '02.

[90]  Alex Samorodnitsky,et al.  Linear Codes and Character Sums , 2002, Comb..

[91]  S. Janson,et al.  RADEMACHER CHAOS: TAIL ESTIMATES VS LIMIT THEOREMS , 2002 .

[92]  Subhash Khot,et al.  On the power of unique 2-prover 1-round games , 2002, Proceedings 17th IEEE Annual Conference on Computational Complexity.

[93]  J. Bourgain.  On the distribution of the fourier spectrum of Boolean functions , 2002 .

[94]  Gil Kalai,et al.  A Fourier-theoretic perspective on the Condorcet paradox and Arrow's theorem , 2002, Adv. Appl. Math..

[95]  Elchanan Mossel,et al.  On the noise sensitivity of monotone functions , 2003, Random Struct. Algorithms.

[96]  Caroline J. Klivans.  Combinatorial properties of shifted complexes , 2003 .

[97]  S. Safra,et al.  Noise-Resistant Boolean-Functions are Juntas , 2003 .

[98]  Ryan O'Donnell,et al.  Learning juntas , 2003, STOC '03.

[99]  Ryan O'Donnell,et al.  Learning DNF from random walks , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[100]  Ronen Shaltiel.  Towards proving strong direct product theorems , 2003, computational complexity.

[101]  Adam R. Klivans,et al.  Learning intersections and thresholds of halfspaces , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[102]  Guy Kindler,et al.  Testing juntas , 2002, J. Comput. Syst. Sci..

[103]  Rocco A. Servedio,et al.  Learning DNF in time 2Õ(n1/3) , 2004, J. Comput. Syst. Sci..

[104]  Hana Chockler,et al.  A lower bound for testing juntas , 2004, Inf. Process. Lett..

[105]  Ehud Friedgut,et al.  Influences in Product Spaces: KKL and BKKKL Revisited , 2004, Combinatorics, Probability and Computing.

[106]  Ryan O'Donnell,et al.  Hardness amplification within NP , 2002, Proceedings 17th IEEE Annual Conference on Computational Complexity.

[107]  Ryan O'Donnell,et al.  Coin flipping from a cosmic source: On error correction of truly random bits , 2004, Random Struct. Algorithms.

[108]  Ran Raz,et al.  Fourier analysis for probabilistic communication complexity , 1995, computational complexity.