Robust Volume Minimization-Based Matrix Factorization for Remote Sensing and Document Clustering

This paper considers volume minimization (VolMin)-based structured matrix factorization. VolMin is a factorization criterion that decomposes a given data matrix into a basis matrix times a structured coefficient matrix via finding the minimum-volume simplex that encloses all the columns of the data matrix. Recent work showed that VolMin guarantees the identifiability of the factor matrices under mild conditions that are realistic in a wide variety of applications. This paper focuses on both theoretical and practical aspects of VolMin. On the theory side, exact equivalence of two independently developed sufficient conditions for VolMin identifiability is proven here, thereby providing a more comprehensive understanding of this aspect of VolMin. On the algorithm side, computational complexity and sensitivity to outliers are two key challenges associated with real-world applications of VolMin. These are addressed here via a new VolMin algorithm that handles volume regularization in a computationally simple way, and automatically detects and iteratively downweights outliers, simultaneously. Simulations and real-data experiments using a remotely sensed hyperspectral image and the Reuters document corpus are employed to showcase the effectiveness of the proposed algorithm.

[1]  Andreas T. Ernst,et al.  ICE: a statistical approach to identifying endmembers in hyperspectral images , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Stephen P. Boyd,et al.  Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices , 2003, Proceedings of the 2003 American Control Conference, 2003..

[3]  Zhi-Quan Luo,et al.  A Unified Algorithmic Framework for Block-Structured Optimization Involving Big Data: With applications in machine learning and signal processing , 2015, IEEE Signal Processing Magazine.

[4]  Paul D. Gader,et al.  A Signal Processing Perspective on Hyperspectral Unmixing , 2014 .

[5]  Nicolas Gillis,et al.  The Why and How of Nonnegative Matrix Factorization , 2014, ArXiv.

[6]  Jiawei Han,et al.  Locally Consistent Concept Factorization for Document Clustering , 2011, IEEE Transactions on Knowledge and Data Engineering.

[7]  Peter Gritzmann,et al.  Largest j-Simplices in n-Polytopes , 1994, Universität Trier, Mathematik/Informatik, Forschungsbericht.

[8]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[9]  Alfred O. Hero,et al.  Nonlinear Unmixing of Hyperspectral Images: Models and Algorithms , 2013, IEEE Signal Processing Magazine.

[10]  José M. Bioucas-Dias,et al.  Self-Dictionary Sparse Regression for Hyperspectral Unmixing: Greedy Pursuit and Pure Pixel Search Are Related , 2014, IEEE Journal of Selected Topics in Signal Processing.

[11]  Nikos D. Sidiropoulos,et al.  Blind Separation of Quasi-Stationary Sources: Exploiting Convex Geometry in Covariance Domain , 2015, IEEE Transactions on Signal Processing.

[12]  L. Ammann Robust Principal Components , 1989 .

[13]  Peter Gritzmann,et al.  Largestj-simplices inn-polytopes , 1995, Discret. Comput. Geom..

[14]  Nicolas Gillis,et al.  Fast and Robust Recursive Algorithmsfor Separable Nonnegative Matrix Factorization , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Nikos D. Sidiropoulos,et al.  Robust volume minimization-based matrix factorization via alternating optimization , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  José M. Bioucas-Dias,et al.  A variable splitting augmented Lagrangian approach to linear spectral unmixing , 2009, 2009 First Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing.

[17]  Mark D. Plumbley,et al.  Theorems on Positive Data: On the Uniqueness of NMF , 2008, Comput. Intell. Neurosci..

[18]  Wei-Chiang Li,et al.  Identifiability of the Simplex Volume Minimization Criterion for Blind Hyperspectral Unmixing: The No-Pure-Pixel Case , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[20]  Antonio J. Plaza,et al.  A Signal Processing Perspective on Hyperspectral Unmixing: Insights from Remote Sensing , 2014, IEEE Signal Processing Magazine.

[21]  Jean-Yves Tourneret,et al.  Nonlinear unmixing of hyperspectral images using a generalized bilinear model , 2011, 2011 IEEE Statistical Signal Processing Workshop (SSP).

[22]  Zhi-Quan Luo,et al.  A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization , 2012, SIAM J. Optim..

[23]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Hairong Qi,et al.  Endmember Extraction From Highly Mixed Data Using Minimum Volume Constrained Nonnegative Matrix Factorization , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Feiping Nie,et al.  Optimal Mean Robust Principal Component Analysis , 2014, ICML.

[26]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[27]  Chong-Yung Chi,et al.  Chance-Constrained Robust Minimum-Volume Enclosing Simplex Algorithm for Hyperspectral Unmixing , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Shiqian Ma,et al.  A smoothing SQP framework for a class of composite Lq\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$L_q$$\end{documen , 2014, Mathematical Programming.

[29]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[30]  Nikolaos D. Sidiropoulos,et al.  Putting nonnegative matrix factorization to the test: a tutorial derivation of pertinent cramer—rao bounds and performance benchmarking , 2014, IEEE Signal Processing Magazine.

[31]  Zhaoshui He,et al.  Minimum-Volume-Constrained Nonnegative Matrix Factorization: Enhanced Ability of Learning Parts , 2011, IEEE Transactions on Neural Networks.

[32]  Chong-Yung Chi,et al.  A Convex Analysis-Based Minimum-Volume Enclosing Simplex Algorithm for Hyperspectral Unmixing , 2009, IEEE Transactions on Signal Processing.

[33]  Yihong Gong,et al.  Document clustering by concept factorization , 2004, SIGIR '04.

[34]  Wotao Yin,et al.  A Block Coordinate Descent Method for Regularized Multiconvex Optimization with Applications to Nonnegative Tensor Factorization and Completion , 2013, SIAM J. Imaging Sci..

[35]  H. Ekblom Lp-methods for robust regression , 1974 .

[36]  Nikos D. Sidiropoulos,et al.  Tensor-based power spectra separation and emitter localization for cognitive radio , 2014, 2014 IEEE 8th Sensor Array and Multichannel Signal Processing Workshop (SAM).

[37]  Narayan Prasad,et al.  On Robust Weighted-Sum Rate Maximization in MIMO Interference Networks , 2011, 2011 IEEE International Conference on Communications (ICC).

[38]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[39]  Nikos D. Sidiropoulos,et al.  Non-Negative Matrix Factorization Revisited: Uniqueness and Algorithm for Symmetric Decomposition , 2014, IEEE Transactions on Signal Processing.

[40]  Miguel Á. Carreira-Perpiñán,et al.  Projection onto the probability simplex: An efficient algorithm with a simple proof, and an application , 2013, ArXiv.

[41]  Victoria Stodden,et al.  When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts? , 2003, NIPS.

[42]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[43]  Nicolas Dobigeon,et al.  Nonlinear Hyperspectral Unmixing With Robust Nonnegative Matrix Factorization , 2014, IEEE Transactions on Image Processing.

[44]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[45]  Nikos D. Sidiropoulos,et al.  Robust iterative fitting of multilinear models , 2005, IEEE Transactions on Signal Processing.

[46]  José M. Bioucas-Dias,et al.  Minimum Volume Simplex Analysis: A Fast Algorithm to Unmix Hyperspectral Data , 2008, IGARSS 2008 - 2008 IEEE International Geoscience and Remote Sensing Symposium.

[47]  Nikos D. Sidiropoulos,et al.  Joint Tensor Factorization and Outlying Slab Suppression With Applications , 2015, IEEE Transactions on Signal Processing.

[48]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .