Simultaneous feature selection and SVM parameter determination in classification of hyperspectral imagery using Ant Colony Optimization

Hyperspectral remote sensing imagery, due to its rich source of spectral information, provides an efficient tool for land cover classifications in complex geographical areas. However, the high-dimensional space of this imagery poses two important challenges in the classification process: the Hughes phenomena and the existence of relevant and redundant features. The robustness of Support Vector Machines (SVM) in high-dimensional space makes them an efficient tool for classifying hyperspectral imagery. However, optimum SVM parameter determination and optimum feature selection are the two optimization issues that strongly effect SVM performance. Traditional optimization algorithms can discover optimum solutions in a limited search space with one local optimum. Nevertheless, in high-dimensional space traditional optimization algorithms usually get trapped in a local optimum, therefore it is necessary to apply meta-heuristic optimization algorithms to obtain near-global optimum solutions. This study evaluates the potential of Ant Colony Optimization (ACO) for determining SVM parameters and selecting features. Results obtained from AVIRIS and ROSIS hyperspectral datasets demonstrate the superior performance of SVM, achieved by simultaneously optimizing SVM parameters and subsets of the input feature. For comparison, the evaluation is also performed by applying it to other meta-heuristic optimization algorithms such as simulated annealing, tabu search, and genetic algorithm. The results demonstrate a better performance of the ACO-based algorithm in regards to improving the classification accuracy and decreasing the size of selected feature subsets.

[1]  Kent Robertson Van Horn,et al.  Design and application , 1967 .

[2]  R. Lunetta,et al.  Remote sensing and Geographic Information System data integration: error sources and research issues , 1991 .

[3]  Daphne Koller,et al.  Toward Optimal Feature Selection , 1996, ICML.

[4]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[5]  Mukesh Kumar Feature Selection for Classification of Hyperspectral Remotely Sensed data using NSGA-II , 2003 .

[6]  Manuel López-Ibáñez,et al.  Ant colony optimization , 2010, GECCO '10.

[7]  Lorenzo Bruzzone,et al.  Classification of hyperspectral remote sensing images with support vector machines , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[8]  P. Groves,et al.  Methodology For Hyperspectral Band Selection , 2004 .

[9]  Pramod K. Varshney,et al.  HYPERSPECTRAL IMAGE CLASSIFICATION USING SUPPORT VECTOR MACHINES: A COMPARISON WITH DECISION TREE AND NEURAL NETWORK CLASSIFIERS , 2005 .

[10]  Joydeep Ghosh,et al.  Investigation of the random forest framework for classification of hyperspectral data , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Hui-Hua Yang,et al.  Ant colony optimization based network intrusion feature selection and detection , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[12]  Lorenzo Bruzzone,et al.  Kernel-based methods for hyperspectral image classification , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[13]  John Fulcher,et al.  Computational Intelligence: An Introduction , 2008, Computational Intelligence: A Compendium.

[14]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Multiclass SVM Model Selection Using Particle Swarm Optimization , 2006, 2006 Sixth International Conference on Hybrid Intelligent Systems (HIS'06).

[15]  Zhang Xue-qin,et al.  Intrusion Detection System Based on Feature Selection and Support Vector Machine , 2006, 2006 First International Conference on Communications and Networking in China.

[16]  William Stafford Noble,et al.  Support vector machine , 2013 .

[17]  Farid Melgani,et al.  Toward an Optimal SVM Classification System for Hyperspectral Remote Sensing Images , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Shiv O. Prasher,et al.  Measuring performance in precision agriculture: CART-A decision tree approach , 2006 .

[19]  Ling Wang,et al.  A Modified Adaptive Chaotic Binary Ant System and Its Application in Chemical Process Fault Diagnosis , 2006, ICNC.

[20]  Cheng-Lung Huang,et al.  A GA-based feature selection and parameters optimizationfor support vector machines , 2006, Expert Syst. Appl..

[21]  M. Noomen,et al.  Hyperspectral reflectance of vegetation affected by underground hydrocarbon gas seepage , 2007 .

[22]  Duc Truong Pham,et al.  Simultaneous Feature Selection and Parameters Optimization for SVM by Immune Clonal Algorithm , 2005, ICNC.

[23]  Hua-chao Yang,et al.  Research into a Feature Selection Method for Hyperspectral Imagery Using PSO and SVM , 2007 .

[24]  Chen-guang Dai,et al.  Support Vector Machine for Classification of Hyperspectral Remote Sensing Imagery , 2007, Fourth International Conference on Fuzzy Systems and Knowledge Discovery (FSKD 2007).

[25]  Chih-Hung Wu,et al.  A real-valued genetic algorithm to optimize the parameters of support vector machine for predicting bankruptcy , 2007, Expert Syst. Appl..

[26]  Chein-I. Chang Hyperspectral Data Exploitation: Theory and Applications , 2007 .

[27]  Wei-Chiang Hong,et al.  Continuous ant colony optimization algorithms in a support vector regression based financial forecasting model , 2007, Third International Conference on Natural Computation (ICNC 2007).

[28]  Ligang Zheng,et al.  Support Vector Regression and Ant Colony Optimization for Combustion Performance of Boilers , 2008, 2008 Fourth International Conference on Natural Computation.

[29]  Shih-Wei Lin,et al.  Particle swarm optimization for parameter determination and feature selection of support vector machines , 2008, Expert Syst. Appl..

[30]  John B. O. Mitchell,et al.  Simultaneous feature selection and parameter optimisation using an artificial ant colony: case study of melting point prediction , 2008, Chemistry Central journal.

[31]  Zne-Jung Lee,et al.  Parameter determination of support vector machine and feature selection using simulated annealing approach , 2008, Appl. Soft Comput..

[32]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Evolutionary tuning of SVM parameter values in multiclass problems , 2008, Neurocomputing.

[33]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[34]  Yingqin Zhang Evolutionary Computation Based Automatic SVM Model Selection , 2008, 2008 Fourth International Conference on Natural Computation.

[35]  Wei Zhang,et al.  ANN Classification of OMIS Hyperspectral Remotely Sensed Imagery: Experiments and Analysis , 2008, 2008 Congress on Image and Signal Processing.

[36]  Robert I. Damper,et al.  Customizing Kernel Functions for SVM-Based Hyperspectral Image Classification , 2008, IEEE Transactions on Image Processing.

[37]  Li Zhuo,et al.  A genetic algorithm based wrapper feature selection method for classification of hyperspectral images using support vector machine , 2008, Geoinformatics.

[38]  Yong Wang,et al.  Feature selection using tabu search with long-term memories and probabilistic neural networks , 2009, Pattern Recognit. Lett..

[39]  Nasser Ghasem-Aghaee,et al.  Text feature selection using ant colony optimization , 2009, Expert Syst. Appl..

[41]  Cheng-Lung Huang,et al.  ACO-based hybrid classification system with feature subset selection and model parameters optimization , 2009, Neurocomputing.

[42]  Jing Zhao,et al.  A Modified Ant Colony Optimization Algorithm for Tumor Marker Gene Selection , 2009, Genom. Proteom. Bioinform..

[43]  Ming-Chi Lee,et al.  Using support vector machine with a hybrid feature selection method to the stock trend prediction , 2009, Expert Syst. Appl..

[44]  Xiaoli Zhang,et al.  An ACO-based algorithm for parameter optimization of support vector machines , 2010, Expert Syst. Appl..

[45]  Oguz Findik,et al.  A comparison of feature selection models utilizing binary particle swarm optimization and genetic algorithm in determining coronary artery disease using support vector machine , 2010, Expert Syst. Appl..

[46]  Sylvain Arlot,et al.  A survey of cross-validation procedures for model selection , 2009, 0907.4728.

[47]  Mark A. Richardson,et al.  An introduction to hyperspectral imaging and its application for security, surveillance and target acquisition , 2010 .

[48]  Michael Bögl,et al.  Metaheuristic Search Concepts: A Tutorial with Applications to Production and Logistics , 2010 .

[49]  Miao Fang,et al.  Hyperion Hyperspectral Remote Sensing Application in Altered Mineral Mapping in East Kunlun of the Qinghai-Tibet Plateau , 2010, 2010 International Conference on Challenges in Environmental Science and Computer Engineering.

[50]  Antonio J. Plaza,et al.  A Quantitative and Comparative Assessment of Unmixing-Based Feature Extraction Techniques for Hyperspectral Image Classification , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.