High-Dimensional Data Modeling Techniques for Detection of Chemical Plumes and Anomalies in Hyperspectral Images and Movies

We briefly review recent progress in techniques for modeling and analyzing hyperspectral images and movies, in particular for detecting plumes of both known and unknown chemicals. For detecting chemicals of known spectrum, we extend the technique of using a single subspace for modeling the background to a “mixture of subspaces” model to tackle more complicated background. Furthermore, we use partial least squares regression on a resampled training set to boost performance. For the detection of unknown chemicals, we view the problem as an anomaly detection problem and use novel estimators with low-sampled complexity for intrinsically low-dimensional data in high dimensions that enable us to model the “normal” spectra and detect anomalies. We apply these algorithms to benchmark datasets made available by the Automated Target Detection program cofunded by NSF, DTRA, and NGA, and compare, when applicable, to current state-of-the-art algorithms, with favorable results.

[1]  Trac D. Tran,et al.  Chemical plume detection in hyperspectral imagery via joint sparse representation , 2012, MILCOM 2012 - 2012 IEEE Military Communications Conference.

[2]  Mauro Maggioni,et al.  Multiscale Estimation of Intrinsic Dimensionality of Data Sets , 2009, AAAI Fall Symposium: Manifold Learning and Its Applications.

[3]  A. Little Estimating the Intrinsic Dimension of High-Dimensional Data Sets: A Multiscale, Geometric Approach , 2011 .

[4]  Andrea L. Bertozzi,et al.  Detection and tracking of gas plumes in LWIR hyperspectral video sequence data , 2013, Defense, Security, and Sensing.

[5]  Guangliang Chen,et al.  Multiscale geometric and spectral analysis of plane arrangements , 2011, CVPR 2011.

[6]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[7]  Heesung Kwon,et al.  Ensemble learning based on multiple kernel learning for hyperspectral chemical plume detection , 2010, Defense + Commercial Sensing.

[8]  Robert S. DiPietro,et al.  Long-Wave Infrared Hyperspectral Remote Sensing of Chemical Clouds: A focus on signal processing approaches , 2014, IEEE Signal Processing Magazine.

[9]  L. Rosasco,et al.  Multiscale geometric methods for data sets I: Multiscale SVD, noise and curvature , 2017 .

[10]  M. Maggioni,et al.  Multiscale Geometric Methods for Data Sets II: Geometric Multi-Resolution Analysis , 2011, 1105.4924.

[11]  A. Fraser,et al.  Characterizing non-Gaussian clutter and detecting weak gaseous plumes in hyperspectral imagery , 2005 .

[12]  Guangliang Chen,et al.  Spectral Curvature Clustering (SCC) , 2009, International Journal of Computer Vision.

[13]  Mauro Maggioni Geometric estimation of probability measures in high-dimensions , 2013, 2013 Asilomar Conference on Signals, Systems and Computers.

[14]  Louis L. Scharf,et al.  Adaptive subspace detectors , 2001, IEEE Trans. Signal Process..

[15]  Gilad Lerman,et al.  Hybrid Linear Modeling via Local Best-Fit Flats , 2010, International Journal of Computer Vision.

[16]  Ashutosh Kumar Singh,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2010 .

[17]  Gary A. Shaw,et al.  Hyperspectral subpixel target detection using the linear mixing model , 2001, IEEE Trans. Geosci. Remote. Sens..

[18]  James Theiler,et al.  Detection of unknown gas-phase chemical plumes in hyperspectral imagery , 2013, Defense, Security, and Sensing.

[19]  Mauro Maggioni,et al.  Dictionary Learning and Non‐Asymptotic Bounds for Geometric Multi‐Resolution Analysis , 2014 .

[20]  Guangliang Chen,et al.  A fast multiscale framework for data in high-dimensions: Measure estimation, anomaly detection, and compressive measurements , 2012, 2012 Visual Communications and Image Processing.