SNMF-Net: Learning a Deep Alternating Neural Network for Hyperspectral Unmixing

Hyperspectral unmixing is recognized as an important tool to learn the constituent materials and corresponding distribution in a scene. The physical spectral mixture model is always important to tackle this problem because of its highly ill-posed nature. In this article, we introduce a linear spectral mixture model (LMM)-based end-to-end deep neural network named SNMF-Net for hyperspectral unmixing. SNMF-Net shares an alternating architecture and benefits from both model-based methods and learning-based methods. On the one hand, SNMF-Net is of high physical interpretability as it is built by unrolling <inline-formula> <tex-math notation="LaTeX">$L_{p}$ </tex-math></inline-formula> sparsity constrained nonnegative matrix factorization (<inline-formula> <tex-math notation="LaTeX">$L_{p}$ </tex-math></inline-formula>-NMF) model belonging to LMM families. On the other hand, all the parameters and submodules of SNMF-Net can be seamlessly linked with the alternating optimization algorithm of <inline-formula> <tex-math notation="LaTeX">$L_{p}$ </tex-math></inline-formula>-NMF and unmixing problem. This enables us to reasonably integrate the prior knowledge on unmixing, the optimization algorithm, and the sparse representation theory into the network for robust learning, so as to improve unmixing. Experimental results on the synthetic and real-world data show the advantages of the proposed SNMF-Net over many state-of-the-art methods.

[1]  David Zhang,et al.  Learning Iteration-wise Generalized Shrinkage–Thresholding Operators for Blind Deconvolution , 2016, IEEE Transactions on Image Processing.

[2]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[3]  Antonio J. Plaza,et al.  Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[4]  Jun Zhou,et al.  Hyperspectral Restoration via $L_0$ Gradient Regularized Low-Rank Tensor Factorization , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Jocelyn Chanussot,et al.  Object recognition in hyperspectral images using Binary Partition Tree representation , 2015, Pattern Recognit. Lett..

[6]  Guolan Lu,et al.  Medical hyperspectral imaging: a review , 2014, Journal of biomedical optics.

[7]  David Zhang,et al.  A Generalized Iterated Shrinkage Algorithm for Non-convex Sparse Coding , 2013, 2013 IEEE International Conference on Computer Vision.

[8]  Bernhard Schölkopf,et al.  Learning to Deblur , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Theodoor Jacques Marie Ruers,et al.  A Dual Stream Network for Tumor Detection in Hyperspectral Images , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[10]  Jun Zhou,et al.  Nonconvex Nonseparable Sparse Nonnegative Matrix Factorization for Hyperspectral Unmixing , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[11]  Jun Zhou,et al.  Hyperspectral Unmixing via $L_{1/2}$ Sparsity-Constrained Nonnegative Matrix Factorization , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Xiaohan Chen,et al.  ALISTA: Analytic Weights Are As Good As Learned Weights in LISTA , 2018, ICLR.

[13]  Hassan Ghassemian,et al.  Hyperspectral Unmixing Using Deep Convolutional Autoencoders in a Supervised Scenario , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[14]  Guangcan Liu,et al.  Differentiable Linearized ADMM , 2019, ICML.

[15]  Yuanchao Su,et al.  DAEN: Deep Autoencoder Networks for Hyperspectral Unmixing , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Weiwei Sun,et al.  Self-Paced Nonnegative Matrix Factorization for Hyperspectral Unmixing , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Vibhav Vineet,et al.  Conditional Random Fields as Recurrent Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[18]  Qian Du,et al.  Hyperspectral Unmixing Using Sparsity-Constrained Deep Nonnegative Matrix Factorization With Total Variation , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Joan Bruna,et al.  Understanding Trainable Sparse Coding via Matrix Factorization , 2016, 1609.00285.

[20]  Hing Cheung So,et al.  Learning Proximal Operator Methods for Nonconvex Sparse Recovery with Theoretical Guarantee , 2020, IEEE Transactions on Signal Processing.

[21]  Liang Zhang,et al.  Subspace Structure Regularized Nonnegative Matrix Factorization for Hyperspectral Unmixing , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[22]  Xiangfeng Wang,et al.  Nonnegative matrix factorization using ADMM: Algorithm and convergence analysis , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[23]  Deyu Meng,et al.  MHF-Net: An Interpretable Deep Network for Multispectral and Hyperspectral Image Fusion , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Yuan Yan Tang,et al.  Matrix-Vector Nonnegative Tensor Factorization for Blind Unmixing of Hyperspectral Imagery , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Licheng Jiao,et al.  Hyperspectral Unmixing via Deep Convolutional Neural Networks , 2018, IEEE Geoscience and Remote Sensing Letters.

[26]  Hairong Qi,et al.  uDAS: An Untied Denoising Autoencoder With Sparsity for Spectral Unmixing , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Deyu Meng,et al.  Nonconvex-Sparsity and Nonlocal-Smoothness-Based Blind Hyperspectral Unmixing , 2019, IEEE Transactions on Image Processing.

[28]  José M. Bioucas-Dias,et al.  Vertex component analysis: a fast algorithm to unmix hyperspectral data , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[29]  Wen Gao,et al.  Maximal Sparsity with Deep Networks? , 2016, NIPS.

[30]  Yonina C. Eldar,et al.  Efficient and Interpretable Deep Blind Image Deblurring Via Algorithm Unrolling , 2020, IEEE Transactions on Computational Imaging.

[31]  Yuan Yan Tang,et al.  Hyperspectral Unmixing via Total Variation Regularized Nonnegative Tensor Factorization , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Johannes R. Sveinsson,et al.  Convolutional Autoencoder for Spectral-Spatial Hyperspectral Unmixing , 2021, IEEE Trans. Geosci. Remote. Sens..

[33]  Yuntao Qian,et al.  Spectral Mixture Model Inspired Network Architectures for Hyperspectral Unmixing , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Xuelong Li,et al.  Manifold Regularized Sparse NMF for Hyperspectral Unmixing , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Jiangtao Peng,et al.  JMnet: Joint Metric Neural Network for Hyperspectral Unmixing , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Liangpei Zhang,et al.  Total Variation Regularized Reweighted Sparse Nonnegative Matrix Factorization for Hyperspectral Unmixing , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Gozde Bozdagi Akar,et al.  EndNet: Sparse AutoEncoder Network for Endmember Extraction and Hyperspectral Unmixing , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[38]  Chein-I Chang,et al.  Fully constrained least squares linear spectral mixture analysis method for material quantification in hyperspectral imagery , 2001, IEEE Trans. Geosci. Remote. Sens..

[39]  Paul Scheunders,et al.  UnDIP: Hyperspectral Unmixing Using Deep Image Prior , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[40]  Rob Heylen,et al.  Fully Constrained Least Squares Spectral Unmixing by Simplex Projection , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Liangpei Zhang,et al.  Spatial Group Sparsity Regularized Nonnegative Matrix Factorization for Hyperspectral Unmixing , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[42]  Xiaoxiao Li,et al.  Deep Learning Markov Random Field for Semantic Segmentation , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Xiaoqiang Lu,et al.  Spectral–Spatial Joint Sparse NMF for Hyperspectral Unmixing , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[44]  Yuntao Qian,et al.  Adaptive ${L}_{\bf 1/2}$ Sparsity-Constrained NMF With Half-Thresholding Algorithm for Hyperspectral Unmixing , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[45]  Zhangyang Wang,et al.  Frank-Wolfe Network: An Interpretable Deep Structure for Non-Sparse Coding , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[46]  Kun Gao,et al.  Hyperspectral Unmixing Using Orthogonal Sparse Prior-Based Autoencoder With Hyper-Laplacian Loss and Data-Driven Outlier Detection , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[47]  Jie Chen,et al.  A Plug-and-Play Priors Framework for Hyperspectral Unmixing , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[48]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[49]  Qi Xie,et al.  A Model-Driven Deep Neural Network for Single Image Rain Removal , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Saeed Gazor,et al.  Smooth and Sparse Regularization for NMF Hyperspectral Unmixing , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[52]  Ming-Hsuan Yang,et al.  $L_0$ -Regularized Intensity and Gradient Prior for Deblurring Text Images and Beyond , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Hairong Qi,et al.  Endmember Extraction From Highly Mixed Data Using Minimum Volume Constrained Nonnegative Matrix Factorization , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[54]  Pier Luigi Dragotti,et al.  Deep Coupled ISTA Network for Multi-Modal Image Super-Resolution , 2020, IEEE Transactions on Image Processing.

[55]  Jun Zhou,et al.  Material Based Object Tracking in Hyperspectral Videos , 2018, IEEE Transactions on Image Processing.