Hybrid ensembles of decision trees and artificial neural networks

Ensemble learning is inspired by the human group decision making process, and it has been found beneficial in various application domains. Decision tree and artificial neural network are two popular types of classification algorithms often used to construct classic ensembles. Recently, researchers proposed to use the mixture of both types to construct hybrid ensembles. However, researchers use decision trees and artificial neural networks together in an ensemble without further discussion. The focus of this paper is on the hybrid ensemble constructed by using decision trees and artificial neural networks simultaneously. The goal of this paper is not only to show that the hybrid ensemble can achieve comparable or even better classification performance, but also to provide an explanation of why it works.

[1]  Xuejing Sun,et al.  Pitch accent prediction using ensemble machine learning , 2002, INTERSPEECH.

[2]  Weida Tong,et al.  Decision Forest: Combining the Predictions of Multiple Independent Decision Tree Models , 2003, J. Chem. Inf. Comput. Sci..

[3]  Yu-Bin Yang,et al.  Lung cancer cell identification based on artificial neural network ensembles , 2002, Artif. Intell. Medicine.

[4]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[5]  Xindong Wu,et al.  Adaptive Informative Sampling for Active Learning , 2010, SDM.

[6]  Raphaël Marée,et al.  Biomedical Image Classification with Random Subwindows and Decision Trees , 2005, CVBIA.

[7]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[8]  Jian Ma,et al.  A hybrid ensemble approach for enterprise credit risk assessment based on Support Vector Machine , 2012, Expert Syst. Appl..

[9]  Daniel T. Larose,et al.  Discovering Knowledge in Data: An Introduction to Data Mining , 2005 .

[10]  Josh C. Bongard,et al.  Exploiting multiple classifier types with active learning , 2009, GECCO.

[11]  Brijesh Verma,et al.  Hybrid ensemble approach for classification , 2011, Applied Intelligence.

[12]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[13]  Ron Kohavi,et al.  Bias Plus Variance Decomposition for Zero-One Loss Functions , 1996, ICML.

[14]  Chang Shu,et al.  Artificial neural network ensembles and their application in pooled flood frequency analysis , 2004 .

[15]  Elif Derya Übeyli,et al.  ECG beat classifier designed by combined neural network model , 2005, Pattern Recognit..

[16]  Fabio Roli,et al.  Design of effective neural network ensembles for image classification purposes , 2001, Image Vis. Comput..

[17]  Jianping Li,et al.  A novel hybrid ensemble learning paradigm for nuclear energy consumption forecasting , 2012 .

[18]  Pedro M. Domingos A Unified Bias-Variance Decomposition for Zero-One and Squared Loss , 2000, AAAI/IAAI.

[19]  Jaideep Srivastava,et al.  An Empirical Study of Applying Ensembles of Heterogeneous Classifiers on Imperfect Data , 2009, PAKDD Workshops.

[20]  Lun-Ping Hung,et al.  A data driven ensemble classifier for credit scoring analysis , 2010, Expert Syst. Appl..

[21]  Ajith Abraham,et al.  An ensemble of neural networks for weather forecasting , 2004, Neural Computing & Applications.

[22]  Joydeep Ghosh,et al.  Multiclassifier Systems: Back to the Future , 2002, Multiple Classifier Systems.

[23]  Manfred M. Fischer,et al.  Neural network ensembles and their application to traffic flow prediction in telecommunications networks , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[24]  David West,et al.  Neural network ensemble strategies for financial decision applications , 2005, Comput. Oper. Res..

[25]  Sung-Bae Cho,et al.  Activity recognition based on wearable sensors using selection/fusion hybrid ensemble , 2011, 2011 IEEE International Conference on Systems, Man, and Cybernetics.