Fast Singular Value Shrinkage With Chebyshev Polynomial Approximation Based on Signal Sparsity

We propose an approximation method for thresholding of singular values using Chebyshev polynomial approximation (CPA). Many signal processing problems require iterative application of singular value decomposition (SVD) for minimizing the rank of a given data matrix with other cost functions and/or constraints, which is called matrix rank minimization. In matrix rank minimization, singular values of a matrix are shrunk by hard-thresholding, soft-thresholding, or weighted soft-thresholding. However, the computational cost of SVD is generally too expensive to handle high-dimensional signals, such as images; hence, in this case, matrix rank minimization requires enormous computation time. In this paper, we leverage CPA to (approximately) manipulate singular values without computing singular values and vectors. The thresholding of singular values is expressed by a multiplication of certain matrices, which is derived from a characteristic of CPA. The multiplication is also efficiently computed using the sparsity of signals. As a result, the computational cost is significantly reduced. Experimental results suggest the effectiveness of our method through several image processing applications based on matrix rank minimization with nuclear norm relaxation in terms of computation time and approximation precision.

[1]  Shunsuke Ono,et al.  Non-local/local image filters using fast eigenvalue filtering , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[2]  Yousef Saad,et al.  A Chebyshev-Davidson Algorithm for Large Symmetric Eigenproblems , 2007, SIAM J. Matrix Anal. Appl..

[3]  John Wright,et al.  RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  W. Fraser,et al.  A Survey of Methods of Computing Minimax and Near-Minimax Polynomial Approximations for Functions of a Single Independent Variable , 1965, JACM.

[5]  Volkan Cevher,et al.  Convexity in Source Separation : Models, geometry, and algorithms , 2013, IEEE Signal Processing Magazine.

[6]  Kun Li,et al.  Foreground–Background Separation From Video Clips via Motion-Assisted Matrix Restoration , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[8]  Jitendra Malik,et al.  Spectral grouping using the Nystrom method , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  B. Recht,et al.  Tensor completion and low-n-rank tensor recovery via convex optimization , 2011 .

[10]  J. McClellan,et al.  A unified approach to the design of optimum FIR linear-phase digital filters , 1973 .

[11]  G. Phillips.  Interpolation and Approximation by Polynomials , 2003 .

[12]  Lei Zhang,et al.  Weighted Nuclear Norm Minimization with Application to Image Denoising , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[14]  Yuichi Tanaka,et al.  Globalized BM3D using fast eigenvalue filtering , 2015, 2015 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[15]  Yousef Saad,et al.  Approximating Spectral Densities of Large Matrices , 2013, SIAM Rev..

[16]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[17]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[18]  Shunsuke Ono,et al.  Image colorization based on ADMM with fast singular value thresholding by Chebyshev polynomial approximation , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[19]  Shunsuke Ono,et al.  Color-Line Regularization for Color Artifact Removal , 2016, IEEE Transactions on Computational Imaging.

[20]  G. Strang.  Introduction to Linear Algebra , 1993 .

[21]  J. Moreau.  Fonctions convexes duales et points proximaux dans un espace hilbertien , 1962 .

[22]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[23]  Yongtian Wang,et al.  Robust Photometric Stereo via Low-Rank Matrix Completion and Recovery , 2010, ACCV.

[24]  Bin Shen,et al.  Online robust image alignment via iterative convex optimization , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Hongyu Zhao,et al.  Low-Rank Modeling and Its Applications in Image Analysis , 2014, ACM Comput. Surv..

[26]  Kung-Sik Chan,et al.  Reduced rank regression via adaptive nuclear norm penalization. , 2012, Biometrika.

[27]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[28]  A. Cetin,et al.  Equiripple FIR filter design by the FFT algorithm , 1997, IEEE Signal Process. Mag..

[29]  L. Rabiner,et al.  FIR digital filter design techniques using weighted Chebyshev approximation , 1975, Proceedings of the IEEE.

[30]  Shunsuke Ono,et al.  Cartoon-Texture Image Decomposition Using Blockwise Low-Rank Texture Characterization , 2014, IEEE Transactions on Image Processing.

[31]  Xiaowei Zhou,et al.  Moving Object Detection by Detecting Contiguous Outliers in the Low-Rank Representation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  P. Yip,et al.  Discrete Cosine Transform: Algorithms, Advantages, Applications , 1990 .

[33]  John G. Proakis,et al.  Digital Signal Processing: Principles, Algorithms, and Applications , 1992 .

[34]  Yoel Shkolnisky,et al.  An Algorithm for Improving Non-Local Means Operators via Low-Rank Approximation , 2014, IEEE Transactions on Image Processing.

[35]  Paolo Favaro,et al.  A closed-form solution to uncalibrated photometric stereo via diffuse maxima , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Yi Ma,et al.  Repairing Sparse Low-Rank Texture , 2012, ECCV.

[37]  Stanley Osher,et al.  A Low Patch-Rank Interpretation of Texture , 2013, SIAM J. Imaging Sci..

[38]  Y. Saad,et al.  Chebyshev acceleration techniques for solving nonsymmetric eigenvalue problems , 1984 .

[39]  A. W. M. van den Enden,et al.  Discrete Time Signal Processing , 1989 .

[40]  Paris Smaragdis,et al.  Singing-voice separation from monaural recordings using robust principal component analysis , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[41]  James W. Davis,et al.  A Multi-transformational Model for Background Subtraction with Moving Cameras , 2014, ECCV.

[42]  Masaaki Ikehara,et al.  Fast and stable least-squares approach for the design of linear phase FIR filters , 1998, IEEE Trans. Signal Process..

[43]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Yasuyuki Matsushita,et al.  Fast randomized Singular Value Thresholding for Nuclear Norm Minimization , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  S. Osher,et al.  Fast Singular Value Thresholding without Singular Value Decomposition , 2013 .

[46]  Yousef Saad,et al.  Polynomial filtering in latent semantic indexing for information retrieval , 2004, SIGIR '04.

[47]  Robert Bregovic,et al.  Multirate Systems and Filter Banks , 2002 .

[48]  Zhihua Zhang,et al.  Colorization by Matrix Completion , 2012, AAAI.

[49]  Zhu Han,et al.  Collaborative spectrum sensing from sparse observations using matrix completion for cognitive radio networks , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[50]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.