GPU Parallel Implementation of Support Vector Machines for Hyperspectral Image Classification

Support vector machine (SVM) is considered as one of the most powerful classifiers for hyperspectral remote sensing images. However, it has high computational cost. In this paper, we propose a novel two-level parallel computing framework to accelerate the SVM-based classification by utilizing CUDA and OpenMP. For a binary SVM classifier, the kernel function is optimized on GPU, and then a second-order working set selection (WSS) procedure is employed and optimized especially for GPU to reduce the cost of communication between GPU and host. In addition to the parallel binary SVM classifier on GPU as dataprocessing level parallelization, a multiclass SVM is addressed by a “one-against-one” approach in OpenMP, and several binary SVM classifiers are run simultaneously to conduct task-level parallelization. The experimental results show that the solver in this framework offered a speedup of 18.5× over the popular LIBSVM software in the training process for data with 200 bands, 13 classes, and 95 597 training samples, and 81.9× in the testing process for data with 103 bands, 9 classes, 1892 support vectors (SVs), and 42 776 testing samples.

[1]  Santiago Velasco-Forero,et al.  Accelerating hyperspectral manifold learning using graphical processing units , 2009, Defense + Commercial Sensing.

[2]  Gustavo Camps-Valls,et al.  Composite kernels for hyperspectral image classification , 2006, IEEE Geoscience and Remote Sensing Letters.

[3]  Yaohua Luo,et al.  Hyperspectral Remote Sensing Classification Processing Parallel Computing Research Based on GPU , 2012, 2012 International Conference on Computer Science and Electronics Engineering.

[4]  Qian Du,et al.  High Performance Computing for Hyperspectral Remote Sensing , 2011, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[5]  Jon Atli Benediktsson,et al.  SVM- and MRF-Based Method for Accurate Classification of Hyperspectral Images , 2010, IEEE Geoscience and Remote Sensing Letters.

[6]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[7]  Johannes R. Sveinsson,et al.  Spectral and spatial classification of hyperspectral data using SVMs and morphological profiles , 2008, 2007 IEEE International Geoscience and Remote Sensing Symposium.

[8]  Qian Du,et al.  Hyperspectral image compression and target detection using nonlinear principal component analysis , 2013, Optics & Photonics - Optical Engineering + Applications.

[9]  Julien Michel,et al.  Remote Sensing Processing: From Multicore to GPU , 2011, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[10]  Hao Chen,et al.  Parallel Acceleration of SAM Algorithm and Performance Analysis , 2013, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[11]  Francisco Tirado,et al.  GPU for Parallel On-Board Hyperspectral Image Processing , 2008, Int. J. High Perform. Comput. Appl..

[12]  Nicholas Wilt,et al.  The CUDA Handbook: A Comprehensive Guide to GPU Programming , 2013 .

[13]  A F Goetz,et al.  Imaging Spectrometry for Earth Remote Sensing , 1985, Science.

[14]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[15]  Sanguthevar Rajasekaran,et al.  Fast GPU algorithms for endmember extraction from hyperspectral images , 2012, 2012 IEEE Symposium on Computers and Communications (ISCC).

[16]  Antonio J. Plaza,et al.  Multi-GPU Implementation of the Minimum Volume Simplex Analysis Algorithm for Hyperspectral Unmixing , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[17]  Antonio J. Plaza,et al.  GPU Implementation of an Automatic Target Detection and Classification Algorithm for Hyperspectral Image Analysis , 2013, IEEE Geoscience and Remote Sensing Letters.

[18]  S. Sathiya Keerthi,et al.  Improvements to Platt's SMO Algorithm for SVM Classifier Design , 2001, Neural Computation.

[19]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[20]  Qian Du,et al.  Unsupervised Hyperspectral Band Selection Using Graphics Processing Units , 2011, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[21]  Lorenzo Bruzzone,et al.  Classification of hyperspectral remote sensing images with support vector machines , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Antonio J. Plaza,et al.  Clusters versus GPUs for Parallel Target and Anomaly Detection in Hyperspectral Images , 2010, EURASIP J. Adv. Signal Process..

[23]  Chih-Jen Lin,et al.  Working Set Selection Using Second Order Information for Training Support Vector Machines , 2005, J. Mach. Learn. Res..

[24]  Shengli Zhou,et al.  Iterative Sparse Channel Estimation and Decoding for Underwater MIMO-OFDM , 2009, OCEANS 2009.

[25]  Jessica A. Faust,et al.  Imaging Spectroscopy and the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) , 1998 .

[26]  Isabelle Guyon,et al.  Comparison of classifier methods: a case study in handwritten digit recognition , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[27]  Lorenzo Bruzzone,et al.  Kernel-based methods for hyperspectral image classification , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[28]  John R. Williams,et al.  Parallel multiclass classification using SVMs on GPUs , 2010, GPGPU-3.

[29]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[30]  Shao-Yi Chien,et al.  Support Vector Machines on GPU with Sparse Matrix Format , 2010, 2010 Ninth International Conference on Machine Learning and Applications.

[31]  Kurt Keutzer,et al.  Fast support vector machine training and classification on graphics processors , 2008, ICML '08.

[32]  Émilie Chouzenoux,et al.  Fast Constrained Least Squares Spectral Unmixing Using Primal-Dual Interior-Point Optimization , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.