Learning Fourier Sparse Set Functions

Can we learn a sparse graph from observing the value of a few random cuts? This and more general problems can be reduced to the challenge of learning set functions known to have sparse Fourier support contained in some collectionP. We prove that if we choose O(k log 4 |P|) sets uniformly at random, then with high probability, observing any k-sparse function on those sets is sucient to recover that function exactly. We further show that other properties, such as symmetry or submodularity imply structure in the Fourier spectrum, which can be exploited to further reduce sample complexity. One interesting special case is that it suces to observe O(|E| log 4 (|V|)) values of a cut function to recover a graph. We demonstrate the eectiveness of our results on two realworld reconstruction problems: graph sketching and obtaining fast approximate surrogates to expensive submodular objective functions.

[1]  Emmanuel J. Candès,et al.  Templates for convex cone problems with applications to sparse signal recovery , 2010, Math. Program. Comput..

[2]  S. Foucart A note on guaranteed sparse recovery via ℓ1-minimization , 2010 .

[3]  Noam Nisan,et al.  Constant depth circuits, Fourier transform, and learnability , 1993, JACM.

[4]  Hanna Mazzawi,et al.  Optimally reconstructing weighted graphs using queries , 2010, SODA '10.

[5]  Sudipto Guha,et al.  Near-optimal sparse fourier representations via sampling , 2002, STOC '02.

[6]  Alexander K. Kelmans,et al.  Multiplicative submodularity of a matrix's principal minor as a function of the set of its rows and some combinatorial applications , 1983, Discret. Math..

[7]  Andreas Krause,et al.  Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies , 2008, J. Mach. Learn. Res..

[8]  Nader H. Bshouty,et al.  Optimal Query Complexity for Reconstructing Hypergraphs , 2010, STACS.

[9]  M. L. Fisher,et al.  An analysis of approximations for maximizing submodular set functions—I , 1978, Math. Program..

[10]  László Lovász,et al.  Submodular functions and convexity , 1982, ISMP.

[11]  Noga Alon,et al.  Learning a hidden matching , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[12]  Stephen J. Wright,et al.  Computational Methods for Sparse Solution of Linear Inverse Problems , 2010, Proceedings of the IEEE.

[13]  Jeong Han Kim,et al.  Optimal query complexity bounds for finding graphs , 2010, Artif. Intell..

[14]  Peter C. Fishburn,et al.  INTERDEPENDENCE AND ADDITIVITY IN MULTIVARIATE, UNIDIMENSIONAL EXPECTED UTILITY TIHEORY* , 1967 .

[15]  Hui Lin,et al.  On fast approximate submodular minimization , 2011, NIPS.

[16]  Maria-Florina Balcan,et al.  Learning submodular functions , 2010, STOC '11.

[17]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[18]  Marc Teboulle,et al.  Interior Gradient and Proximal Methods for Convex and Conic Optimization , 2006, SIAM J. Optim..

[19]  Massimo Fornasier,et al.  Compressive Sensing and Structured Random Matrices , 2010 .

[20]  William J. Cook,et al.  Combinatorial optimization , 1997 .

[21]  Vladimir Grebinski,et al.  Optimal Reconstruction of Graphs under the Additive Model , 1997, Algorithmica.

[22]  A. Schrijver A Course in Combinatorial Optimization , 1990 .

[23]  Kyomin Jung,et al.  Almost Tight Upper Bound for Finding Fourier Coefficients of Bounded Pseudo- Boolean Functions , 2008, COLT.

[24]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[25]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[26]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[27]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[28]  Vladimir Grebinski,et al.  Optimal Reconstruction of Graphs under the Additive Model , 2000, Algorithmica.

[29]  M. Stealey,et al.  High Resolution River Hydraulic and Water Quality Characterization Using Rapidly Deployable Networked Infomechanical Systems (NIMS RD) , 2007 .

[30]  Holger Rauhut,et al.  Compressive Sensing with structured random matrices , 2012 .

[31]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[32]  Vahab S. Mirrokni,et al.  Approximating submodular functions everywhere , 2009, SODA.

[33]  M. Rudelson,et al.  On sparse reconstruction from Fourier and Gaussian measurements , 2008 .