Fusion trees for fast and accurate classification of hyperspectral data with ensembles of $$\gamma$$γ-divergence-based RBF networks

Ensembles of RBF networks trained with $$\gamma$$γ-divergence-based similarity measures can improve classification accuracy of hyperspectral imaging data significantly compared to any single RBF network as well as to RBF ensembles based on the Euclidian distance. So far, the drawback of using classifier ensembles is the need to compute the results of a typically large number of RBF networks prior to combination. In this paper, a modified approach to the fusion of classifier outputs is proposed which is based on decision trees. It is shown for several real-world datasets that a small subset of the RBF networks contributes to the decisions in the average case. Hence, for any decision, a conditional computation of required RBF network outputs yields a significant decrease in the computational costs. Additionally, a selection scheme for subsets of RBF classifiers based on their relevance in the fusion process is proposed. This alternative approach can be used, if analysis requires fixed settings, e.g., to meet time constraints.

[1]  Xiujuan Chen,et al.  Type-2 fuzzy logic-based classifier fusion for support vector machines , 2008, Appl. Soft Comput..

[2]  Thomas Villmann,et al.  Generalized matrix learning vector quantizer for the analysis of spectral data , 2008, ESANN.

[3]  Krzysztof Zaremba,et al.  Improving Performance of Self-Organising Maps with Distance Metric Learning Method , 2012, ICAISC.

[4]  Mohamed A. Deriche,et al.  A New Technique for Combining Multiple Classifiers using The Dempster-Shafer Theory of Evidence , 2002, J. Artif. Intell. Res..

[5]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[6]  Felix Bollenbeck,et al.  Robust classification of the nutrition state in crop plants by hyperspectral imaging and artificial neural networks , 2011, 2011 3rd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[7]  Thomas Villmann,et al.  Relevance Learning in Unsupervised Vector Quantization Based on Divergences , 2011, WSOM.

[8]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[9]  Thomas Villmann,et al.  Divergence-Based Vector Quantization , 2011, Neural Computation.

[10]  Udo Seiffert,et al.  A comparison of late fusion methods for object detection , 2013, 2013 IEEE International Conference on Image Processing.

[11]  Thomas Villmann,et al.  Supervised Neural Gas with General Similarity Measure , 2005, Neural Processing Letters.

[12]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Fabio Roli,et al.  Diversity in Classifier Ensembles: Fertile Concept or Dead End? , 2013, MCS.

[14]  Samuel Kaski,et al.  Learning More Accurate Metrics for Self-Organizing Maps , 2002, ICANN.

[15]  Charu C. Aggarwal,et al.  On the Surprising Behavior of Distance Metrics in High Dimensional Spaces , 2001, ICDT.

[16]  J. Ross Quinlan,et al.  Simplifying decision trees , 1987, Int. J. Hum. Comput. Stud..

[17]  Robert P. W. Duin,et al.  The combining classifier: to train or not to train? , 2002, Object recognition supported by user interaction for service robots.

[18]  Robert Sabourin,et al.  Iterative Boolean combination of classifiers in the ROC space: An application to anomaly detection with HMMs , 2010, Pattern Recognit..

[19]  Thomas Villmann,et al.  Divergence based vector quantization of spectral data , 2010, 2010 2nd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing.

[20]  Thomas Villmann,et al.  Divergence based Learning Vector Quantization , 2010, ESANN.

[21]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[22]  Udo Seiffert,et al.  Beyond Standard Metrics - On the Selection and Combination of Distance Metrics for an Improved Classification of Hyperspectral Data , 2014, WSOM.

[23]  Sanggil Kang,et al.  A fusion neural network classifier for image classification , 2009, Pattern Recognit. Lett..

[24]  Thomas Villmann,et al.  Optimization of Parametrized Divergences in Fuzzy c-Means , 2011, ESANN.

[25]  Thomas Villmann,et al.  Generalized relevance learning vector quantization , 2002, Neural Networks.