Learning pseudo-Boolean k-DNF and submodular functions

We prove that any submodular function f: {0,1}^n -> {0,1,...,k} can be represented as a pseudo-Boolean 2k-DNF formula. Pseudo-Boolean DNFs are a natural generalization of DNF representation for functions with integer range. Each term in such a formula has an associated integral constant. We show that an analog of Hastad's switching lemma holds for pseudo-Boolean k-DNFs if all constants associated with the terms of the formula are bounded. This allows us to generalize Mansour's PAC-learning algorithm for k-DNFs to pseudo-Boolean k-DNFs, and hence gives a PAC-learning algorithm with membership queries under the uniform distribution for submodular functions of the form f:{0,1}^n -> {0,1,...,k}. Our algorithm runs in time polynomial in n, k^{O(k \log k / \epsilon)}, 1/\epsilon and log(1/\delta) and works even in the agnostic setting. The line of previous work on learning submodular functions [Balcan, Harvey (STOC '11), Gupta, Hardt, Roth, Ullman (STOC '11), Cheraghchi, Klivans, Kothari, Lee (SODA '12)] implies only n^{O(k)} query complexity for learning submodular functions in this setting, for fixed epsilon and delta. Our learning algorithm implies a property tester for submodularity of functions f:{0,1}^n -> {0, ..., k} with query complexity polynomial in n for k=O((\log n/ \loglog n)^{1/2}) and constant proximity parameter \epsilon.

[1]  Venkatesh Medabalimi Property Testing Lower Bounds via Communication Complexity , 2012 .

[2]  Peter L. Hammer,et al.  Horn Functions and Submodular Boolean Functions , 1997, Theor. Comput. Sci..

[3]  Pravesh Kothari,et al.  Submodular functions are noise stable , 2012, SODA.

[4]  Maria-Florina Balcan,et al.  Learning submodular functions , 2010, ECML/PKDD.

[5]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[6]  Adam Tauman Kalai,et al.  Agnostically learning decision trees , 2008, STOC.

[7]  Dana Ron,et al.  Property testing and its connection to learning and approximation , 1998, JACM.

[8]  MICHAL PARNAS,et al.  On Testing Convexity and Submodularity , 2002, SIAM J. Comput..

[9]  Jan Vondrák,et al.  Is Submodularity Testable? , 2010, Algorithmica.

[10]  Leonid A. Levin,et al.  A hard-core predicate for all one-way functions , 1989, STOC '89.

[11]  Yishay Mansour,et al.  An O(nlog log n) learning algorithm for DNF under the uniform distribution , 1992, COLT '92.

[12]  Eyal Kushilevitz,et al.  Learning decision trees using the Fourier spectrum , 1991, STOC '91.

[13]  S. Boucheron,et al.  A sharp concentration inequality with applications , 1999, Random Struct. Algorithms.

[14]  Yishay Mansour,et al.  An O(n^(log log n)) Learning Algorithm for DNT under the Uniform Distribution , 1995, J. Comput. Syst. Sci..

[15]  Ryan O'Donnell,et al.  Analysis of Boolean Functions , 2014, ArXiv.

[16]  Johan Håstad,et al.  Almost optimal lower bounds for small depth circuits , 1986, STOC '86.

[17]  Eyal Kushilevitz,et al.  Learning Decision Trees Using the Fourier Sprectrum (Extended Abstract) , 1991, Symposium on the Theory of Computing.

[18]  Dana Ron,et al.  Testing Basic Boolean Formulae , 2002, SIAM J. Discret. Math..

[19]  Tim Roughgarden,et al.  Sketching valuation functions , 2012, SODA.

[20]  László Lovász,et al.  Submodular functions and convexity , 1982, ISMP.

[21]  Vahab S. Mirrokni,et al.  Approximating submodular functions everywhere , 2009, SODA.

[22]  Aaron Roth,et al.  Privately Releasing Conjunctions and the Statistical Query Barrier , 2013, SIAM J. Comput..

[23]  Maria-Florina Balcan,et al.  Learning Valuation Functions , 2011, COLT.

[24]  Peter L. Hammer,et al.  Boolean Functions - Theory, Algorithms, and Applications , 2011, Encyclopedia of mathematics and its applications.

[25]  Deeparnab Chakrabarty,et al.  Testing Coverage Functions , 2012, ICALP.

[26]  David P. Woodruff,et al.  Open Problems in Data Streams, Property Testing, and Related Topics , 2011 .

[27]  Jan Vondrák,et al.  A note on concentration of submodular functions , 2010, ArXiv.

[28]  Supported by Anr grant Tamis On concentration of self-bounding functions , 2009 .

[29]  Ronitt Rubinfeld,et al.  Robust Characterizations of Polynomials with Applications to Program Testing , 1996, SIAM J. Comput..

[30]  P. Beame A switching lemma primer , 1994 .