Building ensemble classifiers using belief functions and OWA operators

A pervasive task in many forms of human activity is classification. Recent interest in the classification process has focused on ensemble classifier systems. These types of systems are based on a paradigm of combining the outputs of a number of individual classifiers. In this paper we propose a new approach for obtaining the final output of ensemble classifiers. The method presented here uses the Dempster–Shafer concept of belief functions to represent the confidence in the outputs of the individual classifiers. The combing of the outputs of the individual classifiers is based on an aggregation process which can be seen as a fusion of the Dempster rule of combination with a generalized form of OWA operator. The use of the OWA operator provides an added degree of flexibility in expressing the way the aggregation of the individual classifiers is performed.

[1]  Ian Witten,et al.  Data Mining , 2000 .

[2]  Francesc Esteva,et al.  Review of Triangular norms by E. P. Klement, R. Mesiar and E. Pap. Kluwer Academic Publishers , 2003 .

[3]  Maria Petrou,et al.  Use of Dempster-Shafer theory to combine classifiers which use different class boundaries , 2003, Pattern Analysis & Applications.

[4]  David G. Stork,et al.  Pattern Classification , 1973 .

[5]  Thomas G. Dietterich Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[6]  Pierre Valin,et al.  Data Fusion for Situation Monitoring, Incident Detection, Alert and Response Management , 2005 .

[7]  L. Zadeh,et al.  Data mining, rough sets and granular computing , 2002 .

[8]  Lotfi A. Zadeh,et al.  A COMPUTATIONAL APPROACH TO FUZZY QUANTIFIERS IN NATURAL LANGUAGES , 1983 .

[9]  E. Mandler,et al.  Combining the Classification Results of Independent Classifiers Based on the Dempster/Shafer Theory of Evidence , 1988 .

[10]  Ronald R. Yager,et al.  On ordered weighted averaging aggregation operators in multicriteria decisionmaking , 1988, IEEE Trans. Syst. Man Cybern..

[11]  R. Suganya,et al.  Data Mining Concepts and Techniques , 2010 .

[12]  Ronald R. Yager,et al.  An extension of the naive Bayesian classifier , 2006, Inf. Sci..

[13]  Hakan Altinçay,et al.  On the independence requirement in Dempster-Shafer theory for combining classifiers providing statistical evidence , 2006, Applied Intelligence.

[14]  Marek Reformat A fuzzy‐based multimodel system for reasoning about the number of software defects , 2005, Int. J. Intell. Syst..

[15]  Thierry Denoeux,et al.  Analysis of evidence-theoretic decision rules for pattern classification , 1997, Pattern Recognit..

[16]  Karl Rihaczek,et al.  1. WHAT IS DATA MINING? , 2019, Data Mining for the Social Sciences.

[17]  Mohamed A. Deriche,et al.  A New Technique for Combining Multiple Classifiers using The Dempster-Shafer Theory of Evidence , 2002, J. Artif. Intell. Res..

[18]  Galina L. Rogova,et al.  Combining the results of several neural network classifiers , 1994, Neural Networks.

[19]  Mübeccel Demirekler,et al.  Speaker identification by combining multiple classifiers using Dempster-Shafer theory of evidence , 2003, Speech Commun..

[20]  Michael J. Pazzani,et al.  Error reduction through learning multiple descriptions , 2004, Machine Learning.

[21]  Ronald R. Yager,et al.  On ordered weighted averaging aggregation operators in multicriteria decision-making , 1988 .

[22]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[23]  Alessandro Saffiotti,et al.  The Transferable Belief Model , 1991, ECSQARU.

[24]  Sargur N. Srihari,et al.  Decision Combination in Multiple Classifier Systems , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Thierry Denoeux A k -Nearest Neighbor Classification Rule Based on Dempster-Shafer Theory , 2008, Classic Works of the Dempster-Shafer Theory of Belief Functions.

[26]  Thierry Denoeux,et al.  A neural network classifier based on Dempster-Shafer theory , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[27]  Elisabetta Binaghi,et al.  Fuzzy Dempster-Shafer reasoning for rule-based classifiers , 1999, Int. J. Intell. Syst..

[28]  IVAN KRAMOSIL Dempster Combination Rule with Boolean-Like Processed Belief Functions , 2001, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[29]  Igor Kononenko,et al.  Learning as Optimization: Stochastic Generation of Multiple Knowledge , 1992, ML.

[30]  Smets Ph.,et al.  Belief functions, Non-standard logics for automated reasoning , 1988 .

[31]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Chris Carter,et al.  Multiple decision trees , 2013, UAI.

[33]  Wray L. Buntine,et al.  A theory of learning classification rules , 1990 .

[34]  Kamal A. Ali A Comparison of Methods for Learning and Combining Evidence From Multiple Models , 1995 .

[35]  R. Yager Quantifier guided aggregation using OWA operators , 1996, Int. J. Intell. Syst..

[36]  Saso Dzeroski,et al.  Combining Multiple Models with Meta Decision Trees , 2000, PKDD.

[37]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[38]  Thierry Denoeux,et al.  An evidence-theoretic k-NN rule with parameter optimization , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[39]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[40]  R. Yager Families of OWA operators , 1993 .

[41]  Nikhil R. Pal,et al.  Land cover classification using fuzzy rules and aggregation of contextual information through evidence theory , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[42]  Hakan Altinçay,et al.  A dempster-shafer theoretic framework for boosting based ensemble design , 2005, Pattern Analysis and Applications.

[43]  Philippe Smets Non-standard logics for automated reasoning , 1988 .

[44]  Radko Mesiar,et al.  Generated triangular norms , 2000, Kybernetika.

[45]  Arthur P. Dempster,et al.  Classic Works on the Dempster-Shafer Theory of Belief Functions (Studies in Fuzziness and Soft Computing) , 2007 .

[46]  B. J. Winer Statistical Principles in Experimental Design , 1992 .

[47]  Ronald R. Yager,et al.  Generalized Naive Bayesian Modeling , 2008 .

[48]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[49]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[50]  Ronald R. Yager,et al.  Extending multicriteria decision making by mixing t‐norms and OWA operators , 2005, Int. J. Intell. Syst..

[51]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[52]  Witold Pedrycz,et al.  Data Mining Methods for Knowledge Discovery , 1998, IEEE Trans. Neural Networks.

[53]  M. O’Hagan A Fuzzy Neuron Based on Maximum Entropy Ordered Weighted Averaging , 1990, 1990 Conference Record Twenty-Fourth Asilomar Conference on Signals, Systems and Computers, 1990..