A Kernel-Based Feature Selection Method for SVM With RBF Kernel for Hyperspectral Image Classification

Hyperspectral imaging fully portrays materials through numerous and contiguous spectral bands. It is a very useful technique in various fields, including astronomy, medicine, food safety, forensics, and target detection. However, hyperspectral images include redundant measurements, and most classification studies encountered the Hughes phenomenon. Finding a small subset of effective features to model the characteristics of classes represented in the data for classification is a critical preprocessing step required to render a classifier effective in hyperspectral image classification. In our previous work, an automatic method for selecting the radial basis function (RBF) parameter (i.e., σ) for a support vector machine (SVM) was proposed. A criterion that contains the between-class and within-class information was proposed to measure the separability of the feature space with respect to the RBF kernel. Thereafter, the optimal RBF kernel parameter was obtained by optimizing the criterion. This study proposes a kernel-based feature selection method with a criterion that is an integration of the previous work and the linear combination of features. In this new method, two properties can be achieved according to the magnitudes of the coefficients being calculated: the small subset of features and the ranking of features. Experimental results on both one simulated dataset and two hyperspectral images (the Indian Pine Site dataset and the Pavia University dataset) show that the proposed method improves the classification performance of the SVM.

[1]  Jon Atli Benediktsson,et al.  Evaluation of Kernels for Multiclass Classification of Hyperspectral Remote Sensing Data , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[2]  John A. Richards,et al.  Efficient maximum likelihood classification for imaging spectrometer data sets , 1994, IEEE Trans. Geosci. Remote. Sens..

[3]  Kevin Leyton-Brown,et al.  SATzilla: Portfolio-based Algorithm Selection for SAT , 2008, J. Artif. Intell. Res..

[4]  Mahesh Pal,et al.  Support vector machine‐based feature selection for land cover classification: a case study with DAIS hyperspectral data , 2006 .

[5]  Ludmila I. Kuncheva,et al.  Using diversity in cluster ensembles , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[6]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[7]  David A. Landgrebe,et al.  Signal Theory Methods in Multispectral Remote Sensing , 2003 .

[8]  Paul Geladi,et al.  Techniques and applications of hyperspectral image analysis , 2007 .

[9]  T. Coleman,et al.  On the Convergence of Reflective Newton Methods for Large-scale Nonlinear Minimization Subject to Bounds , 1992 .

[10]  C. A. Murthy,et al.  Unsupervised Feature Selection Using Feature Similarity , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Gustavo Camps-Valls,et al.  Composite kernels for hyperspectral image classification , 2006, IEEE Geoscience and Remote Sensing Letters.

[12]  Cheng Wang,et al.  Using Stacked Generalization to Combine SVMs in Magnitude and Shape Feature Spaces for Classification of Hyperspectral Data , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Jon Atli Benediktsson,et al.  Spectral–Spatial Classification of Hyperspectral Imagery Based on Partitional Clustering Techniques , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Johannes R. Sveinsson,et al.  Spectral and spatial classification of hyperspectral data using SVMs and morphological profiles , 2008, 2007 IEEE International Geoscience and Remote Sensing Symposium.

[15]  Bernhard Schölkopf,et al.  Remote Sensing Feature Selection by Kernel Dependence Measures , 2010, IEEE Geoscience and Remote Sensing Letters.

[16]  Thomas F. Coleman,et al.  On the convergence of interior-reflective Newton methods for nonlinear minimization subject to bounds , 1994, Math. Program..

[17]  D. Landgrebe Multispectral land sensing: where from, where to? , 2003, IEEE Workshop on Advances in Techniques for Analysis of Remotely Sensed Data, 2003.

[18]  Chein-I. Chang Hyperspectral Data Exploitation: Theory and Applications , 2007 .

[19]  Gustavo Camps-Valls,et al.  Learning Relevant Image Features With Multiple-Kernel Classification , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2004 .

[21]  Johannes R. Sveinsson,et al.  Classification of hyperspectral data from urban areas based on extended morphological profiles , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[23]  P. Gamba,et al.  Exploiting spectral and spatial information for classifying hyperspectral data in urban areas , 2003, IGARSS 2003. 2003 IEEE International Geoscience and Remote Sensing Symposium. Proceedings (IEEE Cat. No.03CH37477).

[24]  Alexander F. H. Goetz,et al.  Three decades of hyperspectral remote sensing of the Earth: a personal view. , 2009 .

[25]  Bor-Chen Kuo,et al.  Feature Mining for Hyperspectral Image Classification , 2013, Proceedings of the IEEE.

[26]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[27]  Chin-Teng Lin,et al.  A Spatial–Contextual Support Vector Machine for Remotely Sensed Image Classification , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[29]  J. Brian Gray,et al.  Applied Regression Including Computing and Graphics , 1999, Technometrics.

[30]  Rick Archibald,et al.  Feature Selection and Classification of Hyperspectral Images With Support Vector Machines , 2007, IEEE Geoscience and Remote Sensing Letters.

[31]  Pao-Ta Yu,et al.  A Dynamic Subspace Method for Hyperspectral Image Classification , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Bor-Chen Kuo,et al.  Kernel Nonparametric Weighted Feature Extraction for Hyperspectral Image Classification , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[33]  William J. Emery,et al.  A neural network approach using multi-scale textural metrics from very high-resolution panchromatic imagery for urban land-use classification , 2009 .

[34]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[35]  William J. Emery,et al.  Classification of Very High Spatial Resolution Imagery Using Mathematical Morphology and Support Vector Machines , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Masoud Nikravesh,et al.  Feature Extraction - Foundations and Applications , 2006, Feature Extraction.

[37]  Antonio J. Plaza,et al.  Dimensionality reduction and classification of hyperspectral image data using sequences of extended morphological transformations , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[38]  Claude Cariou,et al.  BandClust: An Unsupervised Band Reduction Method for Hyperspectral Remote Sensing , 2011, IEEE Geoscience and Remote Sensing Letters.

[39]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[40]  Chein-I Chang,et al.  Hyperspectral Data Exploitation , 2007 .

[41]  Sankar K. Pal,et al.  Pattern Recognition Algorithms for Data Mining , 2004 .

[42]  Bernhard Schölkopf,et al.  Measuring Statistical Dependence with Hilbert-Schmidt Norms , 2005, ALT.

[43]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[44]  Ludmila I. Kuncheva,et al.  Evaluation of Stability of k-Means Cluster Ensembles with Respect to Random Initialization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Chin-Teng Lin,et al.  An Automatic Method for Selecting the Parameter of the Normalized Kernel Function to Support Vector Machines , 2010, 2010 International Conference on Technologies and Applications of Artificial Intelligence.

[46]  Jorge Nocedal,et al.  An interior algorithm for nonlinear optimization that combines line search and trust region steps , 2006, Math. Program..

[47]  Daoqiang Zhang,et al.  Semisupervised Dimensionality Reduction With Pairwise Constraints for Hyperspectral Image Classification , 2011, IEEE Geoscience and Remote Sensing Letters.

[48]  G. F. Hughes,et al.  On the mean accuracy of statistical pattern recognizers , 1968, IEEE Trans. Inf. Theory.

[49]  Bor-Chen Kuo,et al.  Hyperspectral Image Classification Using Kernel-based Nonparametric Weighted Feature Extraction , 2006, 2006 IEEE International Symposium on Geoscience and Remote Sensing.

[50]  David G. Luenberger,et al.  Linear and nonlinear programming , 1984 .

[51]  Bor-Chen Kuo,et al.  Feature Extractions for Small Sample Size Classification Problem , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[52]  Bor-Chen Kuo,et al.  Combining ensemble technique of support vector machines with the optimal kernel method for hyperspectral image classification , 2011, 2011 IEEE International Geoscience and Remote Sensing Symposium.

[53]  Chin-Teng Lin,et al.  An automatic method for selecting the parameter of the RBF kernel function to support vector machines , 2010, 2010 IEEE International Geoscience and Remote Sensing Symposium.

[54]  Lorenzo Bruzzone,et al.  Kernel-based methods for hyperspectral image classification , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[55]  Bor-Chen Kuo,et al.  Nonparametric weighted feature extraction for classification , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[56]  Lorenzo Bruzzone,et al.  Kernel methods for remote sensing data analysis , 2009 .

[57]  Kemal Polat,et al.  Multi-class f-score feature selection approach to classification of obstructive sleep apnea syndrome , 2010, Expert Syst. Appl..

[58]  Lorenzo Bruzzone,et al.  A Novel Context-Sensitive Semisupervised SVM Classifier Robust to Mislabeled Training Samples , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[59]  Lorenzo Bruzzone,et al.  Classification of hyperspectral remote sensing images with support vector machines , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[60]  Jon Atli Benediktsson,et al.  Sensitivity of Support Vector Machines to Random Feature Selection in Classification of Hyperspectral Data , 2010, IEEE Transactions on Geoscience and Remote Sensing.