A stochastic maximum-likelihood framework for simplex structured matrix factorization

Consider a structured matrix factorizaton (SMF) whose coefficient vectors are constrained to lie in the unit simplex. This kind of simplex SMF (SSMF) has received growing attention and has found many applications such as hyperspectral unmixing in remote sensing, text mining in machine learning, and blind source separation in signal processing. The aim of this paper is to establish a maximum-likelihood (ML) estimation framework for SSMF in the presence of Gaussian noise and outliers, and to demonstrate its potential. Our ML formulation has the coefficient vectors marginalized in accordance with a prescribed probabilistic model, and this leads to a likelihood function that contains multi-dimensional integrals. Unfortunately these integrals do not appear to have analytically tractable solutions, and this makes the ML problem challenging. We tackle the problem by using sample average approximation in stochastic optimization and majorization-minimization. Simulation results show that the resulting ML algorithm significantly outperforms several existing methods when noise and outliers are present.

[1]  José M. Bioucas-Dias,et al.  A variable splitting augmented Lagrangian approach to linear spectral unmixing , 2009, 2009 First Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing.

[2]  Nikos D. Sidiropoulos,et al.  Power Spectra Separation via Structured Matrix Factorization , 2016, IEEE Transactions on Signal Processing.

[3]  Nicolas Gillis,et al.  Fast and Robust Recursive Algorithmsfor Separable Nonnegative Matrix Factorization , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Nicolas Gillis,et al.  The Why and How of Nonnegative Matrix Factorization , 2014, ArXiv.

[5]  J. Booth,et al.  Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm , 1999 .

[6]  Nikos D. Sidiropoulos,et al.  Blind Separation of Quasi-Stationary Sources: Exploiting Convex Geometry in Covariance Domain , 2015, IEEE Transactions on Signal Processing.

[7]  Guillermo Sapiro,et al.  See all by looking at a few: Sparse modeling for finding representative objects , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  D. Hunter,et al.  A Tutorial on MM Algorithms , 2004 .

[9]  Paul D. Gader,et al.  A Signal Processing Perspective on Hyperspectral Unmixing , 2014 .

[10]  G. C. Wei,et al.  A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation Algorithms , 1990 .

[11]  Zhi-Quan Luo,et al.  A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization , 2012, SIAM J. Optim..

[12]  Michael Elad,et al.  L1-L2 Optimization in Signal and Image Processing , 2010, IEEE Signal Processing Magazine.

[13]  José M. Bioucas-Dias,et al.  Hyperspectral Unmixing Based on Mixtures of Dirichlet Components , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[14]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[15]  Wei-Chiang Li,et al.  Identifiability of the Simplex Volume Minimization Criterion for Blind Hyperspectral Unmixing: The No-Pure-Pixel Case , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Chong-Yung Chi,et al.  A Convex Analysis-Based Minimum-Volume Enclosing Simplex Algorithm for Hyperspectral Unmixing , 2009, IEEE Transactions on Signal Processing.

[17]  Antonio J. Plaza,et al.  Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[18]  Y. Nesterov Gradient methods for minimizing composite objective function , 2007 .

[19]  Alexander Shapiro,et al.  Lectures on Stochastic Programming: Modeling and Theory , 2009 .

[20]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[21]  Jean-Yves Tourneret,et al.  Nonlinear unmixing of hyperspectral images using a generalized bilinear model , 2011 .

[22]  Sanjeev Arora,et al.  Computing a nonnegative matrix factorization -- provably , 2011, STOC '12.

[23]  Nikos D. Sidiropoulos,et al.  Inexact Alternating Optimization for Phase Retrieval in the Presence of Outliers , 2016, IEEE Transactions on Signal Processing.

[24]  Alfred O. Hero,et al.  Joint Bayesian Endmember Extraction and Linear Unmixing for Hyperspectral Imagery , 2009, IEEE Transactions on Signal Processing.

[25]  Bo Yang,et al.  Robust Volume Minimization-Based Matrix Factorization for Remote Sensing and Document Clustering , 2016, IEEE Transactions on Signal Processing.

[26]  Hairong Qi,et al.  Endmember Extraction From Highly Mixed Data Using Minimum Volume Constrained Nonnegative Matrix Factorization , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Antonio J. Plaza,et al.  A Signal Processing Perspective on Hyperspectral Unmixing: Insights from Remote Sensing , 2014, IEEE Signal Processing Magazine.