Fast General Norm Approximation via Iteratively Reweighted Least Squares

This paper describes an efficient method for general norm approximation that appears frequently in various computer vision problems. Such a lot of problems are differently formulated, but frequently require to minimize the sum of weighted norms as the general norm approximation. Therefore we extend Iteratively Reweighted Least Squares (IRLS) that is originally for minimizing single norm. The proposed method accelerates solving the least-square problem in IRLS by warm start that finds the next solution by the previous solution over iterations. Through numerical tests and application to the computer vision problems, we demonstrate that the proposed method solves the general norm approximation efficiently with small errors.

[1]  Brendt Wohlberg,et al.  An Iteratively Reweighted Norm Algorithm for Minimization of Total Variation Functionals , 2007, IEEE Signal Processing Letters.

[2]  Sivan Toledo,et al.  Blendenpik: Supercharging LAPACK's Least-Squares Solver , 2010, SIAM J. Sci. Comput..

[3]  Srikantan S. Nagarajan,et al.  Iterative Reweighted l1 and l2 Methods for Finding Sparse Solution , 2016 .

[4]  Katsushi Ikeuchi,et al.  3D Shape Restoration via Matrix Recovery , 2010, ACCV Workshops.

[5]  Marc Baboulin,et al.  LU Preconditioning for Overdetermined Sparse Least Squares Problems , 2015, PPAM.

[6]  Per Christian Hansen,et al.  Solution of Ill-Posed Problems by Means of Truncated SVD , 1988 .

[7]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[8]  Kinji Ono,et al.  3D Reconstruction of a Collapsed Historical Site from Sparse Set of Photographs and Photogrammetric Map , 2010, ACCV Workshops.

[9]  Richard I. Hartley,et al.  Convergence of Iteratively Re-weighted Least Squares to Robust M-Estimators , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[10]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[11]  J. Shewchuk.  An Introduction to the Conjugate Gradient Method Without the Agonizing Pain , 1994 .

[12]  Matthew Harker,et al.  Regularized Reconstruction of a Surface from its Measured Gradient Field , 2013, Journal of Mathematical Imaging and Vision.

[13]  Steven J. Benbow,et al.  Solving Generalized Least-Squares Problems with LSQR , 1999, SIAM J. Matrix Anal. Appl..

[14]  Wotao Yin,et al.  Iteratively reweighted algorithms for compressive sensing , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[15]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[16]  Massimo Fornasier,et al.  Iteratively Re-weighted Least Squares minimization: Proof of faster than linear rate for sparse recovery , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[17]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[18]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, STOC '84.

[19]  Bhaskar D. Rao,et al.  Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm , 1997, IEEE Trans. Signal Process..

[20]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, SIGGRAPH 2007.

[21]  Massimo Fornasier,et al.  Conjugate gradient acceleration of iteratively re-weighted least squares methods , 2015, Comput. Optim. Appl..

[22]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[23]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[24]  Richard Szeliski,et al.  Image deblurring and denoising using color priors , 2009, CVPR.

[25]  Yongtian Wang,et al.  Robust Photometric Stereo via Low-Rank Matrix Completion and Recovery , 2010, ACCV.

[26]  C. Sidney Burrus,et al.  Iterative reweighted least-squares design of FIR filters , 1994, IEEE Trans. Signal Process..

[27]  Maryam Fazel,et al.  Iterative reweighted algorithms for matrix rank minimization , 2012, J. Mach. Learn. Res..

[28]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[29]  Junzhou Huang,et al.  Preconditioning for Accelerated Iteratively Reweighted Least Squares in Structured Sparsity Reconstruction , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  James E. Gentle,et al.  Matrix Algebra: Theory, Computations, and Applications in Statistics , 2007 .

[31]  Rama Chellappa,et al.  Enforcing integrability by error correction using l1-minimization , 2009, CVPR.

[32]  J. Rice,et al.  The Lawson algorithm and extensions , 1968 .