Foundations of Ensemble Learning

This chapter describes the basic ideas under the ensemble approach, together with the classical methods that have being used in the field of Machine Learning. Section 3.1 states the rationale under the approach, while in Sect. 3.2 the most popular methods are briefly described. Finally, Sect. 3.3 summarizes and discusses the contents of this chapter.

[1]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[2]  Senén Barro,et al.  Do we need hundreds of classifiers to solve real world classification problems? , 2014, J. Mach. Learn. Res..

[3]  Abbas Mardani,et al.  Multiple criteria decision-making techniques and their applications – a review of the literature from 2000 to 2014 , 2015 .

[4]  José Ramón Quevedo,et al.  Using ensembles for problems with characterizable changes in data distribution: A case study on quantification , 2017, Inf. Fusion.

[5]  Feng Yang,et al.  Robust Feature Selection for Microarray Data Based on Multicriterion Fusion , 2011, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[6]  Marcel Dettling,et al.  BagBoosting for tumor classification with gene expression data , 2004, Bioinform..

[7]  P. Bühlmann,et al.  Boosting With the L2 Loss , 2003 .

[8]  Lawrence O. Hall,et al.  A scalable framework for cluster ensembles , 2009, Pattern Recognit..

[9]  Aurélien Garivier,et al.  On the Complexity of Best-Arm Identification in Multi-Armed Bandit Models , 2014, J. Mach. Learn. Res..

[10]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[11]  Lawrence O. Hall,et al.  A Cluster Ensemble Framework for Large Data sets , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[12]  Yoav Freund,et al.  Boosting a weak learning algorithm by majority , 1995, COLT '90.

[13]  J. Ross Quinlan,et al.  Bagging, Boosting, and C4.5 , 1996, AAAI/IAAI, Vol. 1.

[14]  Thomas L. Saaty.  What is the analytic hierarchy process , 1988 .

[15]  Verónica Bolón-Canedo,et al.  Data classification using an ensemble of filters , 2014, Neurocomputing.

[16]  Kai Ming Ting,et al.  A Comparative Study of Cost-Sensitive Boosting Algorithms , 2000, ICML.

[17]  Edmundas Kazimieras Zavadskas,et al.  Developing a new hybrid MCDM method for selection of the optimal alternative of mechanical longitudinal ventilation of tunnel pollutants during automobile accidents , 2013 .

[18]  Hamido Fujita,et al.  Incremental fuzzy cluster ensemble learning based on rough set theory , 2017, Knowl. Based Syst..

[19]  José Luis Alba-Castro,et al.  Double-base asymmetric AdaBoost , 2013, Neurocomputing.

[20]  Chun-Xia Zhang,et al.  A local boosting algorithm for solving classification problems , 2008, Comput. Stat. Data Anal..

[21]  Gwo-Hshiung Tzeng,et al.  Exploring smart phone improvements based on a hybrid MCDM model , 2014, Expert Syst. Appl..

[22]  Terry Windeatt,et al.  Embedded Feature Ranking for Ensemble MLP Classifiers , 2011, IEEE Transactions on Neural Networks.

[23]  Peter A. Flach,et al.  Machine Learning - The Art and Science of Algorithms that Make Sense of Data , 2012 .

[24]  R. Polikar,et al.  Ensemble based systems in decision making , 2006, IEEE Circuits and Systems Magazine.

[25]  Philip S. Yu,et al.  Top 10 algorithms in data mining , 2007, Knowledge and Information Systems.

[26]  R. Schapire.  The Strength of Weak Learnability , 1990, Machine Learning.

[27]  Verónica Bolón-Canedo,et al.  Ensemble feature selection: Homogeneous and heterogeneous approaches , 2017, Knowl. Based Syst..

[28]  Rich Caruana,et al.  An empirical comparison of supervised learning algorithms , 2006, ICML.

[29]  Verónica Bolón-Canedo,et al.  An ensemble of filters and classifiers for microarray data classification , 2012, Pattern Recognit..

[30]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[31]  Paul A. Viola,et al.  Fast and Robust Classification using Asymmetric AdaBoost and a Detector Cascade , 2001, NIPS.

[32]  Cesare Furlanello,et al.  Parallelizing AdaBoost by weights dynamics , 2007, Comput. Stat. Data Anal..

[33]  J. Skilling.  Bayesian Methods in Cosmology: Foundations and algorithms , 2009 .

[34]  Nikunj C. Oza,et al.  Online Ensemble Learning , 2000, AAAI/IAAI.

[35]  Sandro Vega-Pons,et al.  A Survey of Clustering Ensemble Algorithms , 2011, Int. J. Pattern Recognit. Artif. Intell..

[36]  Joydeep Ghosh,et al.  Cluster Ensembles --- A Knowledge Reuse Framework for Combining Multiple Partitions , 2002, J. Mach. Learn. Res..

[37]  Kurt Hornik,et al.  A Cluster Ensembles Framework , 2003, HIS.

[38]  B. Peter.  BOOSTING FOR HIGH-DIMENSIONAL LINEAR MODELS , 2006 .

[39]  Brendan J. Frey,et al.  Are Random Forests Truly the Best Classifiers? , 2016, J. Mach. Learn. Res..

[40]  Max Bramer,et al.  Principles of Data Mining , 2013, Undergraduate Topics in Computer Science.

[41]  John W. Tukey,et al.  Exploratory data analysis , 1977, Addison-Wesley series in behavioral science : quantitative methods.

[42]  Peter A. Flach,et al.  Cost-sensitive boosting algorithms: Do we really need them? , 2016, Machine Learning.

[43]  Lior Rokach,et al.  Ensemble-based classifiers , 2010, Artificial Intelligence Review.

[44]  Ludmila I. Kuncheva,et al.  Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy , 2003, Machine Learning.

[45]  P. Bühlmann.  Boosting for high-dimensional linear models , 2006 .

[46]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[47]  M. Karthikeyan,et al.  Risk analysis and warning rate of hot environment for foundry industry using hybrid MCDM technique , 2015 .

[48]  Mohammed Attik.  Using Ensemble Feature Selection Approach in Selecting Subset with Relevant Features , 2006, ISNN.

[49]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[50]  Yvan Saeys,et al.  Robust Feature Selection Using Ensemble Feature Selection Techniques , 2008, ECML/PKDD.

[51]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[52]  Xin Yao,et al.  Diversity creation methods: a survey and categorisation , 2004, Inf. Fusion.

[53]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.