Optimizing Subspace SVM Ensemble for Hyperspectral Imagery Classification

In hyperspectral remote sensing image classification, ensemble systems with support vector machine (SVM), such as the Random Subspace SVM Ensemble (RSSE), have significantly outperformed single SVM on the robustness and overall accuracy. In this paper, we introduce a novel subspace mechanism, the Optimizing Subspace SVM Ensemble (OSSE), to improve RSSE by selecting discriminating subspaces for individual SVMs. The framework is based on Genetic Algorithm (GA), adopting the Jeffries-Matusita (JM) distance as a criterion, to optimize the selected subspaces. The combination of optimizing subspaces is more suitable for classification than the random one, at the same time having the ability to accommodate requisite diversity within the ensemble. The modifications have improved the accuracies of individual classifiers; as a result, better overall accuracies are present. Experiments on the classification of two hyperspectral datasets reveal that our proposed OSSE obtains sound performances compared with RSSE, single SVM, and other ensemble with GA to optimize SVM.

[1]  Anil K. Jain,et al.  Feature Selection: Evaluation, Application, and Small Sample Performance , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[3]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[4]  Behnaz Bigdeli,et al.  A Multiple SVM System for Classification of Hyperspectral Remote Sensing Data , 2013, Journal of the Indian Society of Remote Sensing.

[5]  Mineichi Kudo,et al.  Comparison of algorithms that select features for pattern classifiers , 2000, Pattern Recognit..

[6]  J. A. Gualtieri,et al.  Support vector machines for classification of hyperspectral data , 2000, IGARSS 2000. IEEE 2000 International Geoscience and Remote Sensing Symposium. Taking the Pulse of the Planet: The Role of Remote Sensing in Managing the Environment. Proceedings (Cat. No.00CH37120).

[7]  Serkan Günal,et al.  Subspace based feature selection for pattern recognition , 2008, Inf. Sci..

[8]  Giles M. Foody,et al.  Feature Selection for Classification of Hyperspectral Data by SVM , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Lorenzo Bruzzone,et al.  A technique for feature selection in multiclass problems , 2000 .

[10]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[11]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[13]  Jon Atli Benediktsson,et al.  Sensitivity of Support Vector Machines to Random Feature Selection in Classification of Hyperspectral Data , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Joydeep Ghosh,et al.  Investigation of the random forest framework for classification of hyperspectral data , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Lorenzo Bruzzone,et al.  Classification of hyperspectral remote sensing images with support vector machines , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[16]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[17]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[18]  S. Site,et al.  Model for measuring accuracies of majority voting of ensemble classifier with COB and genetic algorithm , 2013, 2013 International Conference on Information Communication and Embedded Systems (ICICES).

[19]  Pao-Ta Yu,et al.  A Dynamic Subspace Method for Hyperspectral Image Classification , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[21]  Liangpei Zhang,et al.  An SVM Ensemble Approach Combining Spectral, Structural, and Semantic Features for the Classification of High-Resolution Remotely Sensed Imagery , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Jefersson Alex dos Santos,et al.  A relevance feedback method based on genetic programming for classification of remote sensing images , 2011, Inf. Sci..

[23]  Lei Tian,et al.  A genetic-algorithm-based selective principal component analysis (GA-SPCA) method for high-dimensional data feature extraction , 2003, IEEE Trans. Geosci. Remote. Sens..

[24]  Giles M. Foody,et al.  A relative evaluation of multiclass image classification by support vector machines , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[25]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[26]  R. Polikar,et al.  Ensemble based systems in decision making , 2006, IEEE Circuits and Systems Magazine.

[27]  Sanghamitra Bandyopadhyay,et al.  Pixel classification using variable string genetic algorithms with chromosome differentiation , 2001, IEEE Trans. Geosci. Remote. Sens..

[28]  Xin Yao,et al.  Ieee Transactions on Knowledge and Data Engineering 1 Relationships between Diversity of Classification Ensembles and Single-class Performance Measures , 2022 .

[29]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[30]  Jack Sklansky,et al.  A note on genetic algorithms for large-scale feature selection , 1989, Pattern Recognit. Lett..

[31]  Paolo Gamba,et al.  Hyperspectral data classification using an ensemble of class-dependent neural networks , 2009, 2009 First Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing.

[32]  Paul M. Mather,et al.  Classification of multisource remote sensing imagery using a genetic algorithm and Markov random fields , 1999, IEEE Trans. Geosci. Remote. Sens..

[33]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[34]  Farid Melgani,et al.  Toward an Optimal SVM Classification System for Hyperspectral Remote Sensing Images , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[35]  John A. Richards,et al.  Remote Sensing Digital Image Analysis , 1986 .

[36]  Philip H. Swain,et al.  Remote Sensing: The Quantitative Approach , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Lorenzo Bruzzone,et al.  An extension of the Jeffreys-Matusita distance to multiclass cases for feature selection , 1995, IEEE Trans. Geosci. Remote. Sens..

[38]  Tong Zhang,et al.  An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods , 2001, AI Mag..

[39]  Heesung Kwon,et al.  Feature-based ensemble learning for hyperspectral chemical plume detection , 2011 .

[40]  David A. Landgrebe,et al.  Hyperspectral image data analysis , 2002, IEEE Signal Process. Mag..

[41]  Joydeep Ghosh,et al.  Best-bases feature extraction algorithms for classification of hyperspectral data , 2001, IEEE Trans. Geosci. Remote. Sens..

[42]  Bogdan Gabrys,et al.  Classifier selection for majority voting , 2005, Inf. Fusion.

[43]  Ching Y. Suen,et al.  Application of majority voting to pattern recognition: an analysis of its behavior and performance , 1997, IEEE Trans. Syst. Man Cybern. Part A.

[44]  Xin-She Yang,et al.  Nature-Inspired Framework for Hyperspectral Band Selection , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[45]  Anderson Rocha,et al.  A framework for selection and fusion of pattern classifiers in multimedia recognition , 2014, Pattern Recognit. Lett..

[46]  Lorenzo Bruzzone,et al.  A new search algorithm for feature selection in hyperspectral remote sensing images , 2001, IEEE Trans. Geosci. Remote. Sens..

[47]  Adrião Duarte Dória Neto,et al.  Random Subspace Method and Genetic Algorithm Applied to a LS-SVM Ensemble , 2012, ICANN.

[48]  David Menotti,et al.  Combining Multiple Classification Methods for Hyperspectral Data Interpretation , 2013, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[49]  Qingfeng Wu,et al.  A Random Feature Selection Approach for Neural Network Ensembles: Considering Diversity , 2010, 2010 International Conference on Computational Intelligence and Software Engineering.

[50]  Gavin Brown,et al.  "Good" and "Bad" Diversity in Majority Vote Ensembles , 2010, MCS.

[51]  Rasmus Fensholt,et al.  Remote Sensing , 2008, Encyclopedia of GIS.

[52]  Nikunj C. Oza,et al.  Online Ensemble Learning , 2000, AAAI/IAAI.

[53]  Qiong Wu,et al.  Ensemble Learning on Hyperspectral Remote Sensing Image Classification , 2012 .

[54]  Lorenzo Bruzzone,et al.  Neuro-fuzzy-combiner: an effective multiple classifier system , 2010, Int. J. Knowl. Eng. Soft Data Paradigms.

[55]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.