Embedded feature selection of hyperspectral bands with boosted decision trees

Feature selection is an important step in hyperspectral analysis using machine learning for many applications, in particular to avoid the curse of dimensionality when there is limited available ground truth. This paper presents an approach to select hyperspectral bands using boosting. Boosting decision trees is an efficient and accurate classification technique that has been applied successfully to process hyperspectral data. The learned structure of the trees can provide insight about which bands are more relevant for the classification. We develop a method that takes into account the improvement obtained by each split of the tree ensemble and calculates a relative importance measure of the input features. The method was evaluated using hyperspectral data of rock samples from an iron ore mine in Australia. We show that by retaining only the most relevant features it is possible to reduce the computational load while retaining classification performance.

[1]  J. Friedman Greedy function approximation: A gradient boosting machine. , 2001 .

[2]  Giles M. Foody,et al.  Feature Selection for Classification of Hyperspectral Data by SVM , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Guy Lapalme,et al.  A systematic analysis of performance measures for classification tasks , 2009, Inf. Process. Manag..

[4]  George C. Runger,et al.  Feature Selection with Ensembles, Artificial Variables, and Redundancy Elimination , 2009, J. Mach. Learn. Res..

[5]  Antonio Torralba,et al.  Sharing Visual Features for Multiclass and Multiview Object Detection , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[7]  Y. Freund,et al.  Discussion of the Paper \additive Logistic Regression: a Statistical View of Boosting" By , 2000 .

[8]  P. Mather,et al.  Classification Methods for Remotely Sensed Data , 2001 .

[9]  Jon Atli Benediktsson,et al.  Recent Advances in Techniques for Hyperspectral Image Processing , 2009 .

[10]  Richard J. Murphy,et al.  Applying boosting for hyperspectral classification of ore-bearing rocks , 2009, 2009 IEEE International Workshop on Machine Learning for Signal Processing.

[11]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[12]  Jason M. Eichenholz,et al.  Ultracompact fully integrated megapixel multispectral imager , 2009, OPTO.

[13]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[14]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[15]  R. T. Clarke,et al.  FEATURE SELECTION BY USING CLASSIFICATION AND REGRESSION TREES ( CART ) , 2004 .