Fast dimensionality reduction and classification of hyperspectral images with extreme learning machines

Recent advances in remote sensing techniques allow for the collection of hyperspectral images with enhanced spatial and spectral resolution. In many applications, these images need to be processed and interpreted in real-time, since analysis results need to be obtained almost instantaneously. However, the large amount of data that these images comprise introduces significant processing challenges. This also complicates the analysis performed by traditional machine learning algorithms. To address this issue, dimensionality reduction techniques aim at reducing the complexity of data while retaining the relevant information for the analysis, removing noise and redundant information. In this paper, we present a new real-time method for dimensionality reduction and classification of hyperspectral images. The newly proposed method exploits artificial neural networks, which are used to develop a fast compressor based on the extreme learning machine. The obtained experimental results indicate that the proposed method has the ability to compress and classify high-dimensional images fast enough for practical use in real-time applications.

[1]  P. Atkinson,et al.  Introduction Neural networks in remote sensing , 1997 .

[2]  C. D. Meyer,et al.  Generalized inverses of linear transformations , 1979 .

[3]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[4]  Guang-Bin Huang,et al.  Extreme Learning Machine for Multilayer Perceptron , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Zongben Xu,et al.  Universal Approximation of Extreme Learning Machine With Adaptive Growth of Hidden Nodes , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[6]  I. Jolliffe.  Principal Component Analysis , 2002 .

[7]  J. L. Buckner.  NASA Advanced Component Technology Program, investments in remote sensing technologies , 2003, IGARSS 2003. 2003 IEEE International Geoscience and Remote Sensing Symposium. Proceedings (IEEE Cat. No.03CH37477).

[8]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[9]  Jon Atli Benediktsson,et al.  Independent Component Discriminant Analysis for hyperspectral image classification , 2010, 2010 2nd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing.

[10]  Antonio Plaza,et al.  A new deep convolutional neural network for fast hyperspectral image classification , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[11]  Antonio J. Plaza,et al.  Real-time implementation of remotely sensed hyperspectral image unmixing on GPUs , 2012, Journal of Real-Time Image Processing.

[12]  Victor C. M. Leung,et al.  Extreme Learning Machines [Trends & Controversies] , 2013, IEEE Intelligent Systems.

[13]  Naif Alajlan,et al.  Differential Evolution Extreme Learning Machine for the Classification of Hyperspectral Images , 2014, IEEE Geoscience and Remote Sensing Letters.

[14]  José M. Bioucas-Dias,et al.  Does independent component analysis play a role in unmixing hyperspectral data? , 2005, IEEE Trans. Geosci. Remote. Sens..

[15]  G. F. Hughes,et al.  On the mean accuracy of statistical pattern recognizers , 1968, IEEE Trans. Inf. Theory.

[16]  Juha Karhunen,et al.  Unsupervised deep learning , 2015 .

[17]  Chi-Man Vong,et al.  Local Receptive Fields Based Extreme Learning Machine , 2015, IEEE Computational Intelligence Magazine.

[18]  Gang Wang,et al.  Deep Learning-Based Classification of Hyperspectral Data , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[19]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[20]  Juha Karhunen,et al.  Chapter 7 – Unsupervised deep learning: A short review , 2015 .

[21]  Zhiqiong Wang,et al.  Elastic extreme learning machine for big data classification , 2015, Neurocomputing.

[22]  Jon Atli Benediktsson,et al.  A comparative study of different ICA algorithms for hyperspectral image analysis , 2013, 2013 5th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[23]  Zhou Jie,et al.  Hyperspectral image classification via kernel extreme learning machine using local receptive fields , 2016 .

[24]  Du Kai,et al.  Using non-negative matrix factorization with projected gradient for hyperspectral images feature extraction , 2013, 2013 IEEE 8th Conference on Industrial Electronics and Applications (ICIEA).

[25]  Geoffrey E. Hinton,et al.  Autoencoders, Minimum Description Length and Helmholtz Free Energy , 1993, NIPS.

[26]  Marco Diani,et al.  Hyperspectral Signal Subspace Identification in the Presence of Rare Signal Components , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[27]  R. Kavitha,et al.  Hyperspectral image analysis techniques on remote sensing , 2017, 2017 Third International Conference on Sensing, Signal Processing and Security (ICSSS).

[28]  Guang-Bin Huang,et al.  What are Extreme Learning Machines? Filling the Gap Between Frank Rosenblatt’s Dream and John von Neumann’s Puzzle , 2015, Cognitive Computation.

[29]  Heng Li,et al.  ELM-based spectral-spatial classification of hyperspectral images using bilateral filtering information on spectral band-subsets , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[30]  Nicolas Gillis,et al.  Sparse nonnegative matrix underapproximation and its application to hyperspectral image analysis , 2011, 2011 3rd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[31]  Antonio J. Plaza,et al.  Fusion of hyperspectral and LiDAR data using morphological component analysis , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[32]  Guang-Bin Huang,et al.  An Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels , 2014, Cognitive Computation.

[33]  Marco Diani,et al.  A New Algorithm for Robust Estimation of the Signal Subspace in Hyperspectral Images in the Presence of Rare Signal Components , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[34]  M. Drazin.  Review: S. L. Campbell and C. D. Meyer, Jr., Generalized inverses of linear transformations , 1980 .

[35]  Inés del Campo,et al.  ELM-based hyperspectral imagery processor for onboard real-time classification , 2016, 2016 Conference on Design and Architectures for Signal and Image Processing (DASIP).

[36]  Peter L. Bartlett,et al.  The Sample Complexity of Pattern Classification with Neural Networks: The Size of the Weights is More Important than the Size of the Network , 1998, IEEE Trans. Inf. Theory.

[37]  Jon Atli Benediktsson,et al.  Statistical methods and neural network approaches for classification of data from multiple sources , 1991 .

[38]  Guang-Bin Huang,et al.  Convex incremental extreme learning machine , 2007, Neurocomputing.

[39]  Chen Chen,et al.  Spectral-Spatial Classification of Hyperspectral Image Based on Kernel Extreme Learning Machine , 2014, Remote. Sens..

[40]  B. Kunkel,et al.  ROSIS (Reflective Optics System Imaging Spectrometer) - A Candidate Instrument For Polar Platform Missions , 1988, Other Conferences.

[41]  K. S. Banerjee.  Generalized Inverse of Matrices and Its Applications , 1973 .

[42]  Yicong Zhou,et al.  Extreme learning machine for ranking: Generalization analysis and applications , 2014, Neural Networks.

[43]  Naif Alajlan,et al.  A hierarchical learning paradigm for semi-supervised classification of remote sensing images , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[44]  Hongming Zhou,et al.  Extreme Learning Machines [Trends & Controversies] , 2013 .

[45]  Yicong Zhou,et al.  Extreme Learning Machine With Composite Kernels for Hyperspectral Image Classification , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[46]  Jun Li,et al.  ${{\rm E}^{2}}{\rm LMs}$ : Ensemble Extreme Learning Machines for Hyperspectral Image Classification , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[47]  Amaury Lendasse,et al.  Extreme learning machines for soybean classification in remote sensing hyperspectral images , 2014, Neurocomputing.

[48]  Cheng Wu,et al.  Discriminative clustering via extreme learning machine , 2015, Neural Networks.

[49]  J. B. Lee,et al.  Enhancement of high spectral resolution remote-sensing data by a noise-adjusted principal components transform , 1990 .

[50]  Jianyu Chen,et al.  Supervised classification of hyperspectral images using local-receptive-fields-based kernel extreme learning machine , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[51]  Jing Wang,et al.  Independent component analysis-based dimensionality reduction with applications in hyperspectral image analysis , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[52]  Cheng Wu,et al.  Semi-Supervised and Unsupervised Extreme Learning Machines , 2014, IEEE Transactions on Cybernetics.

[53]  Antonio J. Plaza,et al.  Fast principal component analysis for hyperspectral imaging based on cloud computing , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[54]  Chee Kheong Siew,et al.  Extreme learning machine: RBF network case , 2004, ICARCV 2004 8th Control, Automation, Robotics and Vision Conference, 2004..

[55]  David Malah,et al.  Rank Estimation and Redundancy Reduction of High-Dimensional Noisy Signals With Preservation of Rare Vectors , 2007, IEEE Transactions on Signal Processing.

[56]  Matthias F. Carlsohn.  Special issue on spectral imaging: Real-time processing of hyperspectral data , 2006, Journal of Real-Time Image Processing.

[57]  Chee Kheong Siew,et al.  Real-time learning capability of neural networks , 2006, IEEE Trans. Neural Networks.

[58]  Tülay Adali,et al.  Spectral–Spatial Classification of Hyperspectral Images Using ICA and Edge-Preserving Filter via an Ensemble Strategy , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[59]  Antonio J. Plaza,et al.  Spectral–Spatial Classification of Hyperspectral Data Using Local and Global Probabilities for Mixed Pixel Characterization , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[60]  Jia Liu,et al.  Change detection based on deep feature representation and mapping transformation for multi-spatial-resolution remote sensing images , 2016 .

[61]  Jocelyn Chanussot,et al.  Spectral compression of hyperspectral images by means of nonlinear principal component analysis decorrelation , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[62]  Chein-I. Chang.  Hyperspectral Imaging: Techniques for Spectral Detection and Classification , 2003 .

[63]  Hongjun Su,et al.  Firefly algorithm optimized extreme learning machine for hyperspectral image classification , 2015, 2015 23rd International Conference on Geoinformatics.

[64]  John A. Richards,et al.  Analysis of remotely sensed data: the formative decades and the future , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[65]  Lei Chen,et al.  Enhanced random search based incremental extreme learning machine , 2008, Neurocomputing.

[66]  Antonio J. Plaza,et al.  Special issue on architectures and techniques for real-time processing of remotely sensed images , 2009, Journal of Real-Time Image Processing.

[67]  Deqin Yan,et al.  Hyperspectral remote sensing image classification with information discriminative extreme learning machine , 2018, Multimedia Tools and Applications.

[68]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[69]  Christos Boutsidis,et al.  SVD based initialization: A head start for nonnegative matrix factorization , 2008, Pattern Recognit..

[70]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[71]  Jun Li,et al.  Advanced Spectral Classifiers for Hyperspectral Images: A review , 2017, IEEE Geoscience and Remote Sensing Magazine.

[72]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[73]  Heng Li,et al.  A Novel Hyperspectral Image Clustering Method With Context-Aware Unsupervised Discriminative Extreme Learning Machine , 2018, IEEE Access.

[74]  P. Switzer,et al.  A transformation for ordering multispectral data in terms of image quality with implications for noise removal , 1988 .

[75]  P. K Varshney,et al.  Advanced image processing techniques for remotely sensed hyperspectral data : with 128 figures and 30 tables , 2004 .

[76]  Daniel Mozos,et al.  FPGA implementation of the principal component analysis algorithm for dimensionality reduction of hyperspectral images , 2019, Journal of Real-Time Image Processing.

[77]  Jessica A. Faust,et al.  Imaging Spectroscopy and the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) , 1998 .

[78]  Qian Du,et al.  Fast real-time onboard processing of hyperspectral imagery for detection and classification , 2009, Journal of Real-Time Image Processing.

[79]  Timothy A. Warner,et al.  Kernel-based extreme learning machine for remote-sensing image classification , 2013 .

[80]  Pierre Comon,et al.  Handbook of Blind Source Separation: Independent Component Analysis and Applications , 2010 .

[81]  Marco Diani,et al.  Hyperspectral Signal Subspace Identification in the Presence of Rare Vectors and Signal-Dependent Noise , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[82]  Qian Du,et al.  Interference and noise-adjusted principal components analysis , 1999, IEEE Trans. Geosci. Remote. Sens..

[83]  A. Kai Qin,et al.  Evolutionary extreme learning machine , 2005, Pattern Recognit..

[84]  Mahesh Pal.  Extreme‐learning‐machine‐based land cover classification , 2008, ArXiv.

[85]  Ignacio Santamaría.  Handbook of Blind Source Separation: Independent Component Analysis and Applications (Common, P. and Jutten, ; 2010 [Book Review] , 2013, IEEE Signal Processing Magazine.

[86]  Antonio J. Plaza,et al.  Parallel Morphological/Neural Classification of Remote Sensing Images Using Fully Heterogeneous and Homogeneous Commodity Clusters , 2006, 2006 IEEE International Conference on Cluster Computing.

[87]  Shin'ichi Tamura,et al.  Capabilities of a four-layered feedforward neural network: four layers versus three , 1997, IEEE Trans. Neural Networks.

[88]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[89]  C. R. Rao,et al.  Generalized Inverse of Matrices and its Applications , 1972 .

[90]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[91]  Jon Atli Benediktsson,et al.  On the use of ICA for hyperspectral image analysis , 2009, 2009 IEEE International Geoscience and Remote Sensing Symposium.

[92]  Jocelyn Chanussot,et al.  Detection of Anomalies Produced by Buried Archaeological Structures Using Nonlinear Principal Component Analysis Applied to Airborne Hyperspectral Image , 2013, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[93]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[94]  Jon Atli Benediktsson,et al.  Classification of Hyperspectral Images by Using Extended Morphological Attribute Profiles and Independent Component Analysis , 2011, IEEE Geoscience and Remote Sensing Letters.

[95]  A F Goetz,et al.  Imaging Spectrometry for Earth Remote Sensing , 1985, Science.

[96]  Antonio J. Plaza,et al.  Onboard payload-data dimensionality reduction , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[97]  Francisco Argüello,et al.  Exploring ELM-based spatial–spectral classification of hyperspectral images , 2014 .

[98]  Dipankar Das,et al.  Enhanced SenticNet with Affective Labels for Concept-Based Opinion Mining , 2013, IEEE Intelligent Systems.

[99]  Jon Atli Benediktsson,et al.  Conjugate-gradient neural networks in classification of multisource and very-high-dimensional remote sensing data , 1993 .

[100]  Jon Atli Benediktsson,et al.  Kernel Principal Component Analysis for the Classification of Hyperspectral Remote Sensing Data over Urban Areas , 2009, EURASIP J. Adv. Signal Process..

[101]  Shifei Ding,et al.  Extreme learning machine and its applications , 2013, Neural Computing and Applications.

[102]  Jon Atli Benediktsson,et al.  Hyperspectral Image Classification With Independent Component Discriminant Analysis , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[103]  Qian Du,et al.  Band selection for hyperspectral image classification using extreme learning machine , 2017, Defense + Security.

[104]  Jon Atli Benediktsson,et al.  A Study on the Effectiveness of Different Independent Component Analysis Algorithms for Hyperspectral Image Classification , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[105]  Hongming Zhou,et al.  Stacked Extreme Learning Machines , 2015, IEEE Transactions on Cybernetics.

[106]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[107]  Jun Miao,et al.  > Replace This Line with Your Paper Identification Number (double-click Here to Edit) < , 2022 .

[108]  Yong Peng,et al.  Orthogonal extreme learning machine for image classification , 2017, Neurocomputing.

[109]  Yan Yang,et al.  Dimension Reduction With Extreme Learning Machine , 2016, IEEE Transactions on Image Processing.

[110]  Kandarpa Kumar Sarma,et al.  Hyperspectral Remote Sensing Classifications: A Perspective Survey , 2016, Trans. GIS.

[111]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[112]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.