Multiclass Fuzzily Weighted Adaptive-Boosting-Based Self-Organizing Fuzzy Inference Ensemble Systems for Classification

Adaptive boosting (AdaBoost) is a widely used technique to construct a stronger ensemble classifier by combining a set of weaker ones. Zero-order fuzzy inference systems (FISs) are very powerful prototype-based predictive models for classification, offering both great prediction precision and high user interpretability. However, the use of zero-order FISs as base classifiers in AdaBoost has not been explored yet. To bridge the gap, in this article, a novel multiclass fuzzily weighted AdaBoost (FWAdaBoost)-based ensemble system with a self-organizing fuzzy inference system (SOFIS) as the ensemble component is proposed. To better incorporate the SOFIS, FWAdaBoost utilizes the confidence scores produced by the SOFIS in both sample weight updating and ensemble output generation, resulting in more accurate classification boundaries and greater prediction precision. Numerical examples on a wide range of benchmark classification problems demonstrate the efficacy of the proposed approach.

[1]  Plamen P. Angelov,et al.  Self-organizing fuzzy inference ensemble system for big streaming data classification , 2021, Knowl. Based Syst..

[2]  Aboozar Taherkhani,et al.  AdaBoost-CNN: An adaptive boosting algorithm for convolutional neural networks to classify multi-class imbalanced datasets using transfer learning , 2020, Neurocomputing.

[3]  Shakhnaz Akhmedova,et al.  Confidence-Based Voting for the Design of Interpretable Ensembles with Fuzzy Systems , 2020, Algorithms.

[4]  Hong-Jie Xing,et al.  Robust AdaBoost based ensemble of one-class support vector machines , 2020, Inf. Fusion.

[5]  Huanyu Zhao,et al.  Nonlinear system modeling using self-organizing fuzzy neural networks for industrial applications , 2020, Applied Intelligence.

[6]  Chuanjiang Li,et al.  A New Fuzzy Logic Classifier Based on Multiscale Permutation Entropy and Its Application in Bearing Fault Diagnosis , 2019, Entropy.

[7]  Zhigang Zeng,et al.  On the Functional Equivalence of TSK Fuzzy Systems to Neural Networks, Mixture of Experts, CART, and Stacking Ensemble Regression , 2019, IEEE Transactions on Fuzzy Systems.

[8]  Xiaowei Gu,et al.  Local optimality of self-organising neuro-fuzzy inference systems , 2019, Inf. Sci..

[9]  Eugene Semenkin,et al.  Investigation of Reference Sample Reduction Methods for Ensemble Output with Fuzzy Logic-Based Systems , 2019, 2019 8th International Congress on Advanced Applied Informatics (IIAI-AAI).

[10]  Ricardo Tanscheit,et al.  SENFIS - Selected Ensemble of Fuzzy Inference Systems , 2019, 2019 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[11]  Plamen P. Angelov,et al.  A Generalized Methodology for Data Analysis , 2018, IEEE Transactions on Cybernetics.

[12]  Hani Hagras,et al.  Toward Human-Understandable, Explainable AI , 2018, Computer.

[13]  Lior Rokach,et al.  Ensemble learning: A survey , 2018, WIREs Data Mining Knowl. Discov..

[14]  Plamen P. Angelov,et al.  Self-organising fuzzy logic classifier , 2018, Inf. Sci..

[15]  Plamen P. Angelov,et al.  A Massively Parallel Deep Rule-Based Ensemble Classifier for Remote Sensing Scenes , 2018, IEEE Geoscience and Remote Sensing Letters.

[16]  Witold Pedrycz,et al.  Evolving Ensemble Fuzzy Classifier , 2017, IEEE Transactions on Fuzzy Systems.

[17]  Zachary Chase Lipton.  The mythos of model interpretability , 2016, ACM Queue.

[18]  Zhu Wang,et al.  Robust boosting with truncated loss functions , 2018 .

[19]  Plamen P. Angelov,et al.  Autonomous learning multi-model classifier of 0-Order (ALMMo-0) , 2017, 2017 Evolving and Adaptive Intelligent Systems (EAIS).

[20]  Bo Sun,et al.  A robust multi-class AdaBoost algorithm for mislabeled noisy data , 2016, Knowl. Based Syst..

[21]  Xin Yao,et al.  Online Ensemble Learning of Data Streams with Gradually Evolved Classes , 2016, IEEE Transactions on Knowledge and Data Engineering.

[22]  Franziska Hoffmann,et al.  Spatial Tessellations Concepts And Applications Of Voronoi Diagrams , 2016 .

[23]  Hiroshi Nagahashi,et al.  Parameterized AdaBoost: Introducing a Parameter to Speed Up the Training of Real AdaBoost , 2014, IEEE Signal Processing Letters.

[24]  Araceli Sanchis,et al.  Ensemble method based on individual evolving classifiers , 2013, 2013 IEEE Conference on Evolving and Adaptive Intelligent Systems (EAIS).

[25]  Sam Kwong,et al.  A noise-detection based AdaBoost algorithm for mislabeled data , 2012, Pattern Recognit..

[26]  Amel Borgi,et al.  An ensemble method for fuzzy rule-based classification systems , 2012, Knowledge and Information Systems.

[27]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[28]  Plamen P. Angelov,et al.  A new type of simplified fuzzy rule-based system , 2012, Int. J. Gen. Syst..

[29]  Rafal Scherer,et al.  An ensemble of logical-type neuro-fuzzy systems , 2011, Expert Syst. Appl..

[30]  Rafal Scherer,et al.  Designing Boosting Ensemble of Relational Fuzzy Systems , 2010, Int. J. Neural Syst..

[31]  Feng Gao,et al.  Edited AdaBoost by weighted kNN , 2010, Neurocomputing.

[32]  Emanuele Trucco,et al.  FABC: Retinal Vessel Segmentation Using AdaBoost , 2010, IEEE Transactions on Information Technology in Biomedicine.

[33]  Joachim M. Buhmann,et al.  The Balanced Accuracy and Its Posterior Distribution , 2010, 2010 20th International Conference on Pattern Recognition.

[34]  Jordi Vitrià,et al.  Weighted Bagging for Graph Based One-Class Classifiers , 2010, MCS.

[35]  Lior Rokach,et al.  Ensemble-based classifiers , 2010, Artificial Intelligence Review.

[36]  Trevor Hastie,et al.  Multi-class AdaBoost ∗ , 2009 .

[37]  Plamen P. Angelov,et al.  Evolving Fuzzy-Rule-Based Classifiers From Data Streams , 2008, IEEE Transactions on Fuzzy Systems.

[38]  Lei Wang,et al.  AdaBoost with SVM-based component classifiers , 2008, Eng. Appl. Artif. Intell..

[39]  David A. Elizondo,et al.  Bankruptcy forecasting: An empirical comparison of AdaBoost and neural networks , 2008, Decis. Support Syst..

[40]  R. Polikar,et al.  Ensemble based systems in decision making , 2006, IEEE Circuits and Systems Magazine.

[41]  Hong Wei,et al.  Face Verification Using GaborWavelets and AdaBoost , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[42]  Kuo-Chen Chou,et al.  Ensemble classifier for protein fold pattern recognition , 2006, Bioinform..

[43]  Alexander Vezhnevets,et al.  ‘ Modest AdaBoost ’ – Teaching AdaBoost to Generalize Better , 2005 .

[44]  Cynthia Rudin,et al.  The Dynamics of AdaBoost: Cyclic Behavior and Convergence of Margins , 2004, J. Mach. Learn. Res..

[45]  Stan Z. Li,et al.  FloatBoost learning and statistical face detection , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Nikunj C. Oza,et al.  AveBoost2: Boosting for Noisy Data , 2004, Multiple Classifier Systems.

[47]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[48]  Thomas G. Dietterich.  An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization , 2000, Machine Learning.

[49]  Yoram Singer,et al.  BoosTexter: A Boosting-based System for Text Categorization , 2000, Machine Learning.

[50]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[51]  J. Friedman.  Stochastic gradient boosting , 2002 .

[52]  Yoshua Bengio,et al.  Boosting Neural Networks , 2000, Neural Computation.

[53]  J. Friedman.  Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[54]  Yoram Singer,et al.  Improved Boosting Algorithms Using Confidence-rated Predictions , 1998, COLT' 98.

[55]  David W. Opitz,et al.  An empirical evaluation of bagging and boosting for artificial neural networks , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[56]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[57]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[58]  F. Wilcoxon,et al.  Individual comparisons of grouped data by ranking methods. , 1946, Journal of economic entomology.