Tighter bound of Sketched Generalized Matrix Approximation

Generalized matrix approximation plays a fundamental role in many machine learning problems, such as CUR decomposition, kernel approximation, and matrix low rank approximation. Especially with today's applications involved in larger and larger dataset, more and more efficient generalized matrix approximation algorithems become a crucially important research issue. In this paper, we find new sketching techniques to reduce the size of the original data matrix to develop new matrix approximation algorithms. Our results derive a much tighter bound for the approximation than previous works: we obtain a $(1+\epsilon)$ approximation ratio with small sketched dimensions which implies a more efficient generalized matrix approximation.

[1]  Anastasios Kyrillidis,et al.  Approximate matrix multiplication with application to linear embeddings , 2014, 2014 IEEE International Symposium on Information Theory.

[2]  Sjoerd Dirksen,et al.  Toward a unified theory of sparse dimensionality reduction in Euclidean space , 2013, STOC.

[3]  Gary L. Miller,et al.  Iterative Row Sampling , 2012, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[4]  Zhihua Zhang,et al.  Improving CUR matrix decomposition and the Nyström approximation via adaptive sampling , 2013, J. Mach. Learn. Res..

[5]  Christos Boutsidis,et al.  Near Optimal Column-Based Matrix Reconstruction , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[6]  Shusen Wang,et al.  Towards More Efficient Symmetric Matrix Sketching and the CUR Matrix Decomposition , 2015 .

[7]  Edith Cohen,et al.  Approximating matrix multiplication for pattern recognition tasks , 1997, SODA '97.

[8]  David P. Woodruff,et al.  Improved Distributed Principal Component Analysis , 2014, NIPS.

[9]  Christos Boutsidis,et al.  Optimal principal component analysis in distributed and streaming models , 2015, STOC.

[10]  David P. Woodruff Sketching as a Tool for Numerical Linear Algebra , 2014, Found. Trends Theor. Comput. Sci..

[11]  A. Rantzer,et al.  On a generalized matrix approximation problem in the spectral norm , 2012 .

[12]  David P. Woodru Sketching as a Tool for Numerical Linear Algebra , 2014 .

[13]  Michael W. Mahoney,et al.  Revisiting the Nystrom Method for Improved Large-scale Machine Learning , 2013, J. Mach. Learn. Res..

[14]  Anatoli Torokhti,et al.  Generalized Rank-Constrained Matrix Approximations , 2007, SIAM J. Matrix Anal. Appl..

[15]  David P. Woodruff,et al.  Low rank approximation and regression in input sparsity time , 2012, STOC '13.

[16]  Petros Drineas,et al.  Fast Monte Carlo Algorithms for Matrices I: Approximating Matrix Multiplication , 2006, SIAM J. Comput..

[17]  S. Muthukrishnan,et al.  Faster least squares approximation , 2007, Numerische Mathematik.

[18]  Tamás Sarlós,et al.  Improved Approximation Algorithms for Large Matrices via Random Projections , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[19]  G. W. Stewart,et al.  Four algorithms for the the efficient computation of truncated pivoted QR approximations to a sparse matrix , 1999, Numerische Mathematik.

[20]  C. Chui,et al.  Article in Press Applied and Computational Harmonic Analysis a Randomized Algorithm for the Decomposition of Matrices , 2022 .