A Hybrid Decision Support Tool - Using Ensemble of Classifiers

In decision support systems a classification problem can be solved by employing one of several methods such as different types of artificial neural networks, decision trees, bayesian classifiers, etc. However, it may happen that certain parts of instances’ space are better predicting by one method than the others. Thus, the decision of which particular method to choose is a complicated problem. A good alternative to choosing only one method is to create a hybrid forecasting system incorporating a number of possible solution methods as components (an ensemble of classifiers). For this purpose, we have implemented a hybrid decision support system that combines a neural net, a decision tree and a bayesian algorithm using a stacking variant methodology. The presented system can be trained with any data, but in the current implementation is mainly used by tutors of Hellenic Open University to identify drop-out prone students. However, a comparison with other ensembles using the same classifiers as base learner on several standard benchmark datasets showed that this tool gives better accuracy in most cases.

[1]  Ian H. Witten,et al.  Induction of model trees for predicting continuous classes , 1996 .

[2]  Bernard Zenko,et al.  Is Combining Classifiers Better than Selecting the Best One , 2002, ICML.

[3]  Cullen Schaffer,et al.  Selecting a classification method by cross-validation , 1993, Machine Learning.

[4]  OpitzDavid,et al.  Popular ensemble methods , 1999 .

[5]  Yong Wang,et al.  Using Model Trees for Classification , 1998, Machine Learning.

[6]  Chuanyi Ji,et al.  Combinations of Weak Classifiers , 1996, NIPS.

[7]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[8]  D. Opitz,et al.  Popular Ensemble Methods: An Empirical Study , 1999, J. Artif. Intell. Res..

[9]  Ian Witten,et al.  Data Mining , 2000 .

[10]  Thomas G. Dietterich.  Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[11]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[12]  Steven Salzberg,et al.  On Comparing Classifiers: Pitfalls to Avoid and a Recommended Approach , 1997, Data Mining and Knowledge Discovery.

[13]  Johannes Fürnkranz,et al.  An Evaluation of Grading Classifiers , 2001, IDA.

[14]  Efraim Turban,et al.  Decision support systems and intelligent systems , 1997 .

[15]  Yoav Freund,et al.  Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[16]  Ian H. Witten,et al.  Issues in Stacked Generalization , 2011, J. Artif. Intell. Res..

[17]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[18]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[19]  Alexander K. Seewald,et al.  How to Make Stacking Better and Faster While Also Taking Care of an Unknown Weakness , 2002, International Conference on Machine Learning.

[20]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[21]  Sotiris B. Kotsiantis,et al.  Preventing Student Dropout in Distance Learning Using Machine Learning Techniques , 2003, KES.

[22]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[23]  Eric Bauer,et al.  An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants , 1999, Machine Learning.

[24]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.