Toward dynamic ensembles: the BAGA approach

Summary form only given. Existing studies on ensemble classifiers typically take a static approach in assembling individual classifiers, in which all the important features are specified in advance. In this paper, we propose a new concept, dynamic ensemble, as an advanced classifier that could have dynamic component classifiers and have dynamic configurations. Toward this goal, we have substantially expanded the existing "overproduce and choose" paradigm for ensemble construction. A new algorithm called BAGA is proposed to explore this approach. Taking a set of decision tree component classifiers as input, BAGA generates a set of candidate ensembles using combined bagging and genetic algorithm techniques so that component classifiers are determined at execution time. Empirical studies have been carried out on variations of the BAGA algorithm, where the sizes of chosen classifiers, effects of bag size, voting function and evaluation functions on the dynamic ensemble construction, are investigated.

[1]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[2]  Anders Krogh,et al.  Neural Network Ensembles, Cross Validation, and Active Learning , 1994, NIPS.

[3]  Fabio Roli,et al.  Methods for Designing Multiple Classifier Systems , 2001, Multiple Classifier Systems.

[4]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[5]  Amanda J. C. Sharkey,et al.  Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems , 1999 .

[6]  D. Wolpert.  Combining Generalizers Using Partitions of the Learning Set , 1993 .

[7]  Fabio Roli,et al.  Design of Multiple Classifier Systems , 2002 .

[8]  J. Ross Quinlan,et al.  Bagging, Boosting, and C4.5 , 1996, AAAI/IAAI, Vol. 1.

[9]  Thomas G. Dietterich.  Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[10]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Fabio Roli,et al.  An approach to the automatic design of multiple classifier systems , 2001, Pattern Recognit. Lett..

[12]  Nitesh V. Chawla,et al.  Generalization Methods in Bioinformatics , 2002, BIOKDD.

[13]  Fabio Roli,et al.  Intrusion detection in computer networks by multiple classifier systems , 2002, Object recognition supported by user interaction for service robots.

[14]  D. Opitz,et al.  Popular Ensemble Methods: An Empirical Study , 1999, J. Artif. Intell. Res..

[15]  Dennis Bahler,et al.  Combining Heterogeneous Sets of Classifiers : Theoretical and Experimental Comparison of Methods , 2000 .

[16]  Berkman Sahiner,et al.  Dual system approach to computer-aided detection of breast masses on mammograms. , 2006, Medical physics.

[17]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[18]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[19]  João Gama,et al.  Combining Classifiers by Constructive Induction , 1998, ECML.

[20]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[21]  Thomas G. Dietterich.  Machine-Learning Research Four Current Directions , 1997 .

[22]  Leo Breiman,et al.  Randomizing Outputs to Increase Prediction Accuracy , 2000, Machine Learning.