Dictionary Learning for Noisy and Incomplete Hyperspectral Images

We consider analysis of noisy and incomplete hyperspectral imagery, with the objective of removing the noise and inferring the missing data. The noise statistics may be wavelength dependent, and the fraction of data missing (at random) may be substantial, including potentially entire bands, offering the potential to significantly reduce the quantity of data that need be measured. To achieve this objective, the imagery is divided into contiguous three-dimensional (3D) spatio-spectral blocks of spatial dimension much less than the image dimension. It is assumed that each such 3D block may be represented as a linear combination of dictionary elements of the same dimension, plus noise, and the dictionary elements are learned in situ based on the observed data (no a priori training). The number of dictionary elements needed for representation of any particular block is typically small relative to the block dimensions, and all the image blocks are processed jointly (“collaboratively") to infer the underlying dictionary. We address dictionary learning from a Bayesian perspective, considering two distinct means of imposing sparse dictionary usage. These models allow inference of the number of dictionary elements needed as well as the underlying wavelength-dependent noise statistics. It is demonstrated that drawing the dictionary elements from a Gaussian process prior, imposing structure on the wavelength dependence of the dictionary elements, yields significant advantages, relative to the more conventional approach of using an independent and identically distributed Gaussian prior for the dictionary elements; this advantage is particularly evident in the presence of noise. The framework is demonstrated by processing hyperspectral imagery with a significant number of voxels missing uniformly at random, with imagery at specific wavelengths missing entirely, and in the presence of substantial additive noise.

[1]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[2]  Guillermo Sapiro,et al.  Online dictionary learning for sparse coding , 2009, ICML '09.

[3]  Lawrence Carin,et al.  Exploiting Structure in Wavelet-Based Bayesian Compressive Sensing , 2009, IEEE Transactions on Signal Processing.

[4]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[5]  Paul D. Gader,et al.  Vegetation Mapping for Landmine Detection Using Long-Wave Hyperspectral Imagery , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[6]  W. Yin,et al.  HYPERSPECTRAL DATA RECONSTRUCTION COMBINING SPATIAL AND SPECTRAL SPARSITY , 2010 .

[7]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[8]  Michael Elad,et al.  Sparse Representation for Color Image Restoration , 2008, IEEE Transactions on Image Processing.

[9]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[10]  G. Casella,et al.  The Bayesian Lasso , 2008 .

[11]  H. Zou The Adaptive Lasso and Its Oracle Properties , 2006 .

[12]  Michael Elad,et al.  Learning Multiscale Sparse Representations for Image and Video Restoration , 2007, Multiscale Model. Simul..

[13]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[14]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[15]  Zoubin Ghahramani,et al.  Infinite Sparse Factor Analysis and Infinite Independent Components Analysis , 2007, ICA.

[16]  Paul D. Gader,et al.  Hyperspectral Band Selection and Endmember Detection Using Sparsity Promoting Priors , 2008, IEEE Geoscience and Remote Sensing Letters.

[17]  Michael Elad,et al.  A Weighted Average of Sparse Representations is Better than the Sparsest One Alone , 2008, Structured Decompositions and Efficient Algorithms.

[18]  Guillermo Sapiro,et al.  Spatially Coherent Nonlinear Dimensionality Reduction and Segmentation of Hyperspectral Images , 2007, IEEE Geoscience and Remote Sensing Letters.

[19]  Jean-Michel Morel,et al.  Self-Similarity Driven Color Demosaicking , 2009, IEEE Transactions on Image Processing.

[20]  Thomas L. Griffiths,et al.  Infinite latent feature models and the Indian buffet process , 2005, NIPS.

[21]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[22]  Volkan Cevher,et al.  Compressible Distributions for High-Dimensional Statistics , 2011, IEEE Transactions on Information Theory.

[23]  Michael Möller,et al.  A Convex Model for Matrix Factorization and Dimensionality Reduction on Physical Space and Its Application to Blind Hyperspectral Unmixing , 2010 .

[24]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[25]  Chein-I Chang,et al.  Unsupervised hyperspectral image analysis with projection pursuit , 2000, IEEE Trans. Geosci. Remote. Sens..

[26]  Yasuyuki Matsushita,et al.  High-resolution hyperspectral imaging via matrix factorization , 2011, CVPR 2011.

[27]  Guillermo Sapiro,et al.  Learning to Sense Sparse Signals: Simultaneous Sensing Matrix and Sparsifying Dictionary Optimization , 2009, IEEE Transactions on Image Processing.

[28]  Guillermo Sapiro,et al.  Multiscale Representation and Segmentation of Hyperspectral Imagery Using Geometric Partial Differential Equations and Algebraic Multigrid Methods , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[29]  J. Bobin,et al.  Sparsity constraints for hyperspectral data analysis: linear mixture model and beyond , 2009, Optical Engineering + Applications.

[30]  Matthew West,et al.  Bayesian factor regression models in the''large p , 2003 .

[31]  Begüm Demir,et al.  Clustering-Based Extraction of Border Training Patterns for Accurate SVM Classification of Hyperspectral Images , 2009, IEEE Geoscience and Remote Sensing Letters.

[32]  Bruno A. Olshausen,et al.  Learning Sparse Codes for Hyperspectral Imagery , 2011, IEEE Journal of Selected Topics in Signal Processing.

[33]  V. P. Pauca,et al.  Nonnegative matrix factorization for spectral data analysis , 2006 .

[34]  Rajat Raina,et al.  Self-taught learning: transfer learning from unlabeled data , 2007, ICML '07.

[35]  Stanley Osher,et al.  A split Bregman method for non-negative sparsity penalized least squares with applications to hyperspectral demixing , 2010, 2010 IEEE International Conference on Image Processing.

[36]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[37]  John B. Greer,et al.  Sparse Demixing of Hyperspectral Images , 2012, IEEE Transactions on Image Processing.

[38]  Volkan Cevher,et al.  Sparse Signal Recovery Using Markov Random Fields , 2008, NIPS.

[39]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[40]  Guillermo Sapiro,et al.  Supervised Dictionary Learning , 2008, NIPS.

[41]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[42]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[43]  Lawrence Carin,et al.  Nonparametric factor analysis with beta process priors , 2009, ICML '09.

[44]  Marc'Aurelio Ranzato,et al.  Efficient Learning of Sparse Representations with an Energy-Based Model , 2006, NIPS.

[45]  J. B. Lee,et al.  Enhancement of high spectral resolution remote-sensing data by a noise-adjusted principal components transform , 1990 .

[46]  Bruno A. Olshausen,et al.  Learning Horizontal Connections in a Sparse Coding Model of Natural Images , 2007, NIPS.

[47]  Guillermo Sapiro,et al.  Discriminative sparse representations in hyperspectral imagery , 2010, 2010 IEEE International Conference on Image Processing.

[48]  Yonina C. Eldar,et al.  Exploiting Statistical Dependencies in Sparse Representations for Signal Recovery , 2010, IEEE Transactions on Signal Processing.

[49]  Guillermo Sapiro,et al.  Non-Parametric Bayesian Dictionary Learning for Sparse Image Representations , 2009, NIPS.

[50]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[51]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[53]  Michael I. Jordan,et al.  Hierarchical Beta Processes and the Indian Buffet Process , 2007, AISTATS.

[54]  Jean-Luc Starck,et al.  Dictionary learning with spatio-spectral sparsity constraints , 2009 .

[55]  Paul D. Gader,et al.  Sparsity Promoting Iterated Constrained Endmember Detection in Hyperspectral Imagery , 2007, IEEE Geoscience and Remote Sensing Letters.