Multiresolution Matrix Factorization

Large matrices appearing in machine learning problems often have complex hierarchical structures that go beyond what can be found by traditional linear algebra tools, such as eigendecomposition. Inspired by ideas from multiresolution analysis, this paper introduces a new notion of matrix factorization that can capture structure in matrices at multiple different scales. The resulting Multiresolution Matrix Factorizations (MMFs) not only provide a wavelet basis for sparse approximation, but can also be used for matrix compression (similar to Nystrom approximations) and as a prior for matrix completion.

[1]  W MahoneyMichael,et al.  Fast Monte Carlo Algorithms for Matrices III , 2006 .

[2]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[3]  P. Visscher,et al.  GCTA: a tool for genome-wide complex trait analysis. , 2011, American journal of human genetics.

[4]  Petros Drineas,et al.  FAST MONTE CARLO ALGORITHMS FOR MATRICES II: COMPUTING A LOW-RANK APPROXIMATION TO A MATRIX∗ , 2004 .

[5]  Michael W. Mahoney.  Randomized Algorithms for Matrices and Data , 2011, Found. Trends Mach. Learn..

[6]  Y. Meyer,et al.  Harmonic Analysis on Spaces of Homogeneous Type , 2008 .

[7]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[8]  Dimitris Achlioptas,et al.  Fast computation of low rank matrix approximations , 2001, STOC '01.

[9]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[10]  A. Cohen.  Ten Lectures on Wavelets, CBMS-NSF Regional Conference Series in Applied Mathematics, Vol. 61, I. Daubechies, SIAM, 1992, xix + 357 pp. , 1994 .

[11]  Ann B. Lee,et al.  Treelets--An adaptive multi-scale basis for sparse unordered data , 2007, 0707.0481.

[12]  K. Taira.  Proof of Theorem 1.3 , 2004 .

[13]  Kathryn Roeder,et al.  REFINING GENETICALLY INFERRED RELATIONSHIPS USING TREELET COVARIANCE SMOOTHING. , 2012, The annals of applied statistics.

[14]  W. Zachary,et al.  An Information Flow Model for Conflict and Fission in Small Groups , 1977, Journal of Anthropological Research.

[15]  Inderjit S. Dhillon,et al.  Clustered low rank approximation of graphs in information science applications , 2011, SDM.

[16]  Inderjit S. Dhillon,et al.  Weighted Graph Cuts without Eigenvectors A Multilevel Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Arthur D. Szlam,et al.  Diffusion wavelet packets , 2006 .

[18]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[19]  Ming-Deh A. Huang,et al.  Proof of proposition 2 , 1992 .

[20]  Ameet Talwalkar,et al.  Sampling Methods for the Nyström Method , 2012, J. Mach. Learn. Res..

[21]  Achi Brandt,et al.  Lean Algebraic Multigrid (LAMG): Fast Graph Laplacian Linear Solver , 2011, SIAM J. Sci. Comput..

[22]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[24]  C. Jacobi,et al.  C. G. J. Jacobi's Gesammelte Werke: Über ein leichtes Verfahren, die in der Theorie der Sacularstorungen vorkommenden Gleichungen numerisch aufzulosen , 1846 .

[25]  Ronald R. Coifman,et al.  Multiscale Wavelets on Trees, Graphs and High Dimensional Data: Theory and Applications to Semi Supervised Learning , 2010, ICML.

[26]  R. Coifman,et al.  Diffusion Wavelets , 2004 .

[27]  C. Jacobi.  Über ein leichtes Verfahren die in der Theorie der Säcularstörungen vorkommenden Gleichungen numerisch aufzulösen*). , 2022 .

[28]  U. Feige,et al.  Spectral Graph Theory , 2015 .

[29]  Leslie Greengard,et al.  A fast algorithm for particle simulations , 1987 .

[30]  Qiang Yang,et al.  Contextual Collaborative Filtering via Hierarchical Matrix Factorization , 2012, SDM.

[31]  Christos Faloutsos,et al.  Graph evolution: Densification and shrinking diameters , 2006, TKDD.

[32]  G. MallatS..  A Theory for Multiresolution Signal Decomposition , 1989 .

[33]  Michael W. Mahoney,et al.  Revisiting the Nystrom Method for Improved Large-scale Machine Learning , 2013, J. Mach. Learn. Res..

[34]  Francis R. Bach,et al.  Structured Sparse Principal Component Analysis , 2009, AISTATS.

[35]  Christos Faloutsos,et al.  Kronecker Graphs: An Approach to Modeling Networks , 2008, J. Mach. Learn. Res..

[36]  J. Edmonds.  Paths, Trees, and Flowers , 1965, Canadian Journal of Mathematics.