Accelerated Search for Non-Negative Greedy Sparse Decomposition via Dimensionality Reduction

Non-negative signals form an important class of sparse signals. Many algorithms have already been proposed to recover such non-negative representations, where greedy and convex relaxed algorithms are among the most popular methods. One fast implementation is the FNNOMP algorithm that updates the non-negative coefficients in an iterative manner. Even though FNNOMP is a good approach when working on libraries of small size, the operational time of the algorithm grows significantly when the size of the library is large. This is mainly due to the selection step of the algorithm that relies on matrix vector multiplications. We here introduce the Embedded Nearest Neighbor (E-NN) algorithm which accelerates the search over large datasets while it is guaranteed to find the most correlated atoms. We then replace the selection step of FNNOMP by E-NN. Furthermore we introduce the Update Nearest Neighbor (U-NN) at the look up table of FNNOMP in order to assure the non-negativity criteria of FNNOMP. The results indicate that the proposed methodology can accelerate FNNOMP with a factor 4 on a real dataset of Raman Spectra and with a factor of 22 on a synthetic dataset.

[1]  Chinmay Hegde,et al.  NuMax: A Convex Approach for Learning Near-Isometric Linear Embeddings , 2015, IEEE Transactions on Signal Processing.

[2]  Di Wu,et al.  A sparse regularized model for Raman spectral analysis , 2014, 2014 Sensor Signal Processing for Defence (SSPD).

[3]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[4]  Piotr Indyk,et al.  Approximate nearest neighbors: towards removing the curse of dimensionality , 1998, STOC '98.

[5]  Antonio J. Plaza,et al.  Sparse Unmixing of Hyperspectral Data , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[6]  B. Moore.  Principal component analysis in linear systems: Controllability, observability, and model reduction , 1981 .

[7]  Konstantinos Themelis,et al.  Bayesian signal processing techniques for hyperspectral image unmixing , 2012 .

[8]  Carlos H. Muravchik,et al.  Enhanced Sparse Bayesian Learning via Statistical Thresholding for Signals in Structured Noise , 2013, IEEE Transactions on Signal Processing.

[9]  John Langford,et al.  Cover trees for nearest neighbor , 2006, ICML.

[10]  Dimitris Achlioptas,et al.  Database-friendly random projections , 2001, PODS.

[11]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[12]  Stephen R. Marsland,et al.  Machine Learning - An Algorithmic Perspective , 2009, Chapman and Hall / CRC machine learning and pattern recognition series.

[13]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[14]  Jun Zhou,et al.  Hyperspectral Unmixing via $L_{1/2}$ Sparsity-Constrained Nonnegative Matrix Factorization , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Mike E. Davies,et al.  Fast Non-Negative Orthogonal Matching Pursuit , 2015, IEEE Signal Processing Letters.

[16]  Jirí Matousek,et al.  On variants of the Johnson–Lindenstrauss lemma , 2008, Random Struct. Algorithms.

[17]  Hyunsoo Kim,et al.  Sparse Non-negative Matrix Factorizations via Alternating Non-negativity-constrained Least Squares , 2006 .

[18]  Jon Louis Bentley,et al.  An Algorithm for Finding Best Matches in Logarithmic Expected Time , 1977, TOMS.