Heterogenous committees with competence analysis

We explore some new types of committees in search of hybrid models successful in many different classification benchmarks. To provide a reliable comparison of the ensembles we restrict the task to some constant configuration of committee members for each benchmark. We were looking for new types of committees which, in such configuration, would be as much accurate and stable as possible. The paper focuses on some ideas of heterogenous committees with different ways of their members' competence estimation. Heterogenous committee members adapt in different ways and are able to solve different problems. Measuring the competence of committee members helps in making competent and accurate decisions.

[1]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[2]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[3]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[4]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[5]  Richard Maclin,et al.  Boosting Classifiers Regionally , 1998, AAAI/IAAI.

[6]  Włodzisław Duch,et al.  Competent Undemocratic Committees , 2003 .

[7]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[8]  Yoav Freund,et al.  Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[9]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[10]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[11]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[12]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[13]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[14]  L. Breiman.  Bias-variance, regularization, instability and stabilization , 1998 .

[15]  Norbert Jankowski,et al.  Mining for Complex Models Comprising Feature Selection and Classification , 2006, Feature Extraction.

[16]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[17]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[18]  David G. Stork,et al.  Pattern Classification , 1973 .

[19]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[20]  Bernard Zenko,et al.  A comparison of stacking with meta decision trees to bagging, boosting, and stacking with other methods , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[21]  Johannes Fürnkranz,et al.  An Evaluation of Grading Classifiers , 2001, IDA.

[22]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[23]  Ian H. Witten,et al.  Stacking Bagged and Dagged Models , 1997, ICML.

[24]  Wodzisaw Duch,et al.  THE SEPARABILITY OF SPLIT VALUE CRITERION , 2000 .

[25]  Ian H. Witten,et al.  Stacked generalization: when does it work? , 1997, IJCAI 1997.

[26]  Leo Breiman,et al.  Stacked regressions , 2004, Machine Learning.

[27]  S. Sathiya Keerthi,et al.  Improvements to Platt's SMO Algorithm for SVM Classifier Design , 2001, Neural Computation.

[28]  K. Gr,et al.  A general purpose separability criterion for classification systems , 1999 .

[29]  Ludmila I. Kuncheva,et al.  Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy , 2003, Machine Learning.