Beyond Moore-Penrose: Sparse pseudoinverse

Frequently, we use the Moore-Penrose pseudoinverse (MPP) even in cases when we do not require all of its defining properties. But if the running time and the storage size are critical, we can do better. By discarding some constraints needed for the MPP, we gain freedom to optimize other aspects of the new pseudoinverse. A sparser pseudoinverse reduces the amount of computation and storage. We propose a method to compute a sparse pseudoinverse and show that it offers sizable improvements in speed and storage, with a small loss in the least-squares performance. Differently from previous approaches, we do not attempt to approximate the MPP, but rather to produce an exact but sparse pseudoinverse. In the underdetermined (compressed sensing) scenario we prove that the rescaled sparse pseudoinverse yields an unbiased estimate of the unknown vector, and we demonstrate its potential in iterative sparse recovery algorithms, pointing out directions for future research.

[1]  S.M. Kay,et al.  Digital signal processing for sonar , 1981, Proceedings of the IEEE.

[2]  S. Kay.  Fundamentals of statistical signal processing: estimation theory , 1993 .

[3]  Ümit V. Çatalyürek,et al.  Hypergraph-Partitioning-Based Decomposition for Parallel Sparse-Matrix Vector Multiplication , 1999, IEEE Trans. Parallel Distributed Syst..

[4]  Pascale Fung,et al.  Sparse Inverse Covariance Matrices for Low Resource Speech Recognition , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[5]  Aleksandar Dogandzic,et al.  Sparse Signal Reconstruction via ECME Hard Thresholding , 2012, IEEE Transactions on Signal Processing.

[6]  Eun Im,et al.  Optimizing the Performance of Sparse Matrix-Vector Multiplication , 2000 .

[7]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[8]  Steven Kay,et al.  Fundamentals Of Statistical Signal Processing , 2001 .

[9]  Jun Zhang,et al.  Sparse inverse preconditioning of multilevel fast multipole algorithm for hybrid Integral equations in electromagnetics , 2004 .

[10]  Arian Maleki,et al.  Optimally Tuned Iterative Reconstruction Algorithms for Compressed Sensing , 2009, IEEE Journal of Selected Topics in Signal Processing.

[11]  Mike E. Davies,et al.  Iterative Hard Thresholding and L0 Regularisation , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[12]  Andrea Montanari,et al.  Universality in Polytope Phase Transitions and Message Passing Algorithms , 2012, ArXiv.

[13]  F. Teixeira,et al.  Differential Forms, Galerkin Duality, and Sparse Inverse Approximations in Finite Element Solutions of Maxwell Equations , 2007, IEEE Transactions on Antennas and Propagation.

[14]  Pascale Fung,et al.  Lowresource speech recognition with automatically learned sparse inverse covariance matrices , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[15]  Jeff A. Bilmes,et al.  Factored sparse inverse covariance matrices , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[16]  R. Vershynin,et al.  A Randomized Kaczmarz Algorithm with Exponential Convergence , 2007, math/0702226.

[17]  A. Samsonov,et al.  Reconstruction of MRI Data Using Sparse Matrix Inverses , 2007, 2007 Conference Record of the Forty-First Asilomar Conference on Signals, Systems and Computers.

[18]  Yong Huat Chew,et al.  Transmitter Precoder Design to Improve the Performance of the MUSIC Algorithm , 2012 .

[19]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[20]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[21]  T. Cui,et al.  The approximate inverse preconditioner for finite element analysis of helmholtz equations , 2009, 2009 Asia Pacific Microwave Conference.