A Convex Model for Nonnegative Matrix Factorization and Dimensionality Reduction on Physical Space

A collaborative convex framework for factoring a data matrix X into a nonnegative product AS , with a sparse coefficient matrix S, is proposed. We restrict the columns of the dictionary matrix A to coincide with certain columns of the data matrix X, thereby guaranteeing a physically meaningful dictionary and dimensionality reduction. We use l1, ∞ regularization to select the dictionary from the data and show that this leads to an exact convex relaxation of l0 in the case of distinct noise-free data. We also show how to relax the restriction-to-X constraint by initializing an alternating minimization approach with the solution of the convex model, obtaining a dictionary close to but not necessarily in X. We focus on applications of the proposed framework to hyperspectral endmember and abundance identification and also show an application to blind source separation of nuclear magnetic resonance data.

[1]  J. Moreau Proximité et dualité dans un espace hilbertien , 1965 .

[2]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[3]  Per Christian Hansen,et al.  Some Applications of the Rank Revealing QR Factorization , 1992, SIAM J. Sci. Comput..

[4]  Mario Winter,et al.  N-FINDR: an algorithm for fast autonomous spectral end-member determination in hyperspectral data , 1999, Optics & Photonics.

[5]  Xin Liu,et al.  Document clustering based on non-negative matrix factorization , 2003, SIGIR.

[6]  José M. Bioucas-Dias,et al.  Vertex component analysis: a fast algorithm to unmix hyperspectral data , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Wady Naanaa,et al.  Blind source separation of positive and partially correlated data , 2005, Signal Process..

[8]  V. P. Pauca,et al.  Nonnegative matrix factorization for spectral data analysis , 2006 .

[9]  J. Tropp Algorithms for simultaneous sparse approximation. Part II: Convex relaxation , 2006, Signal Process..

[10]  D. Brie,et al.  Separation of Non-Negative Mixture of Non-Negative Sources Using a Bayesian Approach and MCMC Sampling , 2006, IEEE Transactions on Signal Processing.

[11]  José M. Bioucas-Dias,et al.  Blind hyperspectral unmixing , 2007, SPIE Remote Sensing.

[12]  Michael W. Berry,et al.  Algorithms and applications for approximate nonnegative matrix factorization , 2007, Comput. Stat. Data Anal..

[13]  Hairong Qi,et al.  Endmember Extraction From Highly Mixed Data Using Minimum Volume Constrained Nonnegative Matrix Factorization , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Yannis Stylianou,et al.  Musical Genre Classification Using Nonnegative Matrix Factorization-Based Features , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[15]  Alina Zare Hyperspectral Endmember Detection and Band Selection Using Bayesian Methods , 2008 .

[16]  Stanley Osher,et al.  L1 unmixing and its application to hyperspectral image enhancement , 2009, Defense + Commercial Sensing.

[17]  Christos Boutsidis,et al.  An improved approximation algorithm for the column subset selection problem , 2008, SODA.

[18]  Alfred O. Hero,et al.  Joint Bayesian Endmember Extraction and Linear Unmixing for Hyperspectral Imagery , 2009, IEEE Transactions on Signal Processing.

[19]  Yong Yu,et al.  Robust Subspace Segmentation by Low-Rank Representation , 2010, ICML.

[20]  John B. Greer Sparse demixing , 2010, Defense + Commercial Sensing.

[21]  Xiaoming Yuan,et al.  A splitting method for separate convex programming with linking linear constraints , 2010 .

[22]  W. Bajwa,et al.  Column Subset Selection with Missing Data , 2010 .

[23]  Andreas Krause,et al.  Submodular Dictionary Selection for Sparse Representation , 2010, ICML.

[24]  Stanley Osher,et al.  A split Bregman method for non-negative sparsity penalized least squares with applications to hyperspectral demixing , 2010, 2010 IEEE International Conference on Image Processing.

[25]  Yonina C. Eldar,et al.  Collaborative hierarchical sparse modeling , 2010, 2010 44th Annual Conference on Information Sciences and Systems (CISS).

[26]  Guillermo Sapiro,et al.  Learning discriminative sparse models for source separation and mapping of hyperspectral imagery , 2010 .

[27]  Ernie Esser,et al.  Applications of Lagrangian-Based Alternating Direction Methods and Connections to Split Bregman , 2009 .

[28]  Julien Mairal,et al.  Network Flow Algorithms for Structured Sparsity , 2010, NIPS.

[29]  Stanley Osher,et al.  A Unified Primal-Dual Algorithm Framework Based on Bregman Iteration , 2010, J. Sci. Comput..

[30]  A. J. Shaka,et al.  Postprocessing and sparse blind source separation of positive and partially overlapped data , 2011, Signal Process..

[31]  Guillermo Sapiro,et al.  Learning Discriminative Sparse Representations for Modeling, Source Separation, and Mapping of Hyperspectral Imagery , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Francis R. Bach,et al.  Structured Variable Selection with Sparsity-Inducing Norms , 2009, J. Mach. Learn. Res..

[33]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[34]  Yonina C. Eldar,et al.  C-HiLasso: A Collaborative Hierarchical Sparse Modeling Framework , 2010, IEEE Transactions on Signal Processing.

[35]  Abhimanyu Das,et al.  Submodular meets Spectral: Greedy Algorithms for Subset Selection, Sparse Approximation and Dictionary Selection , 2011, ICML.