Parallel and Distributed Local Fisher Discriminant Analysis to Reduce Hyperspectral Images on Cloud Computing Architectures

Hyperspectral images are data cubes that offer very rich spectral and spatial resolutions. These images are so highly dimensioned that we generally reduce them in a pre-processing step in order to process them efficiently. In this context, Local Fisher Discriminant Analysis (LFDA) is a feature extraction technique that proved better than several commonly used dimensionality reduction techniques. However, this method suffers from memory problems and long execution times on commodity hardware. In this paper, to solve these problems, we first added an optimization step to LFDA to make it executable on commodity hardware and to make it suitable for parallel and distributed computing, then, we implemented it in a parallel and distributed way using Apache Spark. We tested our implementation on Amazon Web Services (AWS)’s Elastic MapReduce (EMR) clusters, using different hyperspectral images with different sizes. This proved higher performances with a speedup of up to 70x.

[1]  Eric Jones,et al.  SciPy: Open Source Scientific Tools for Python , 2001 .

[2]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[3]  Yuan Li,et al.  Palmprint Recognition Based on Local Fisher Discriminant Analysis , 2014, J. Softw..

[4]  Sergio A. Velastin,et al.  Local Fisher Discriminant Analysis for Pedestrian Re-identification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Antonio J. Plaza,et al.  A New Cloud Computing Architecture for the Classification of Remote Sensing Data , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[6]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[7]  Raphael C.-W. Phan,et al.  Facial Expression Recognition in the Encrypted Domain Based on Local Fisher Discriminant Analysis , 2013, IEEE Transactions on Affective Computing.

[8]  Antonio J. Plaza,et al.  Parallel and Distributed Dimensionality Reduction of Hyperspectral Data on Cloud Computing Architectures , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[9]  Yuefen Chen,et al.  Spoken emotion recognition using local Fisher discriminant analysis , 2010, IEEE 10th INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING PROCEEDINGS.

[10]  Masashi Sugiyama,et al.  Dimensionality Reduction of Multimodal Labeled Data by Local Fisher Discriminant Analysis , 2007, J. Mach. Learn. Res..

[11]  Travis E. Oliphant,et al.  Python for Scientific Computing , 2007, Computing in Science & Engineering.

[12]  Scott Shenker,et al.  Fast and Interactive Analytics over Hadoop Data with Spark , 2012, login Usenix Mag..

[13]  James E. Fowler,et al.  Hyperspectral Image Classification Using Gaussian Mixture Models and Markov Random Fields , 2014, IEEE Geoscience and Remote Sensing Letters.

[14]  Zhihui Wei,et al.  Cloud implementation of hyperspectral image restoration with PCA and total variation based on Spark , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[15]  Lemao Liu,et al.  Local fisher discriminant analysis for spoken language identification , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Ezzeddine Zagrouba,et al.  Impact of Feature Extraction and Feature Selection Techniques on Extended Attribute Profile-based Hyperspectral Image Classification , 2017, VISIGRAPP.

[17]  Gang Wang,et al.  A new hybrid method based on local fisher discriminant analysis and support vector machines for hepatitis disease diagnosis , 2011, Expert Syst. Appl..

[18]  Antonio J. Plaza,et al.  Cloud implementation of the K-means algorithm for hyperspectral image analysis , 2016, The Journal of Supercomputing.

[19]  James E. Fowler,et al.  Locality-Preserving Dimensionality Reduction and Classification for Hyperspectral Image Analysis , 2012, IEEE Transactions on Geoscience and Remote Sensing.