Accuracy weighted diversity-based online boosting

Abstract Target distributional change occurring in a data stream known as concept drift, causes a challenging task for an online learning method, as the accuracy of an online learning method may decrease due to these changes. In this paper, the Accuracy Weighted Diversity-based Online Boosting (AWDOB) method has been proposed, which is based on Adaptable Diversity-based Online Boosting (ADOB) and, other modifications. More precisely, AWDOB uses the proposed accuracy weighting scheme which is based on previous expert’s results of the sums of correctly classified and incorrectly classified instances to calculate the weight of current expert, which improved the overall accuracy of the AWDOB. Experiments were conducted to compare the accuracy results of AWDOB against other methods using ten real-world datasets and thirty-two artificial datasets. Artificial datasets were generated by the four artificial data generators which included gradual and abrupt concept drifts within them. Experimental results suggest that AWDOB beats the accuracy results of other tested methods.

[1]  William Nick Street,et al.  A streaming ensemble algorithm (SEA) for large-scale classification , 2001, KDD '01.

[2]  Mykola Pechenizkiy,et al.  An Overview of Concept Drift Applications , 2016 .

[3]  João Gama,et al.  On evaluating stream learning algorithms , 2012, Machine Learning.

[4]  Cao Feng,et al.  STATLOG: COMPARISON OF CLASSIFICATION ALGORITHMS ON LARGE REAL-WORLD PROBLEMS , 1995 .

[5]  José del Campo-Ávila,et al.  Online and Non-Parametric Drift Detection Methods Based on Hoeffding’s Bounds , 2015, IEEE Transactions on Knowledge and Data Engineering.

[6]  Yu Sun,et al.  Concept Drift Adaptation by Exploiting Historical Knowledge , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Geoff Holmes,et al.  MOA: Massive Online Analysis , 2010, J. Mach. Learn. Res..

[8]  Alberto Cano,et al.  Kappa Updated Ensemble for drifting data stream mining , 2019, Machine Learning.

[9]  Nathalie Japkowicz,et al.  Adaptive learning on mobile network traffic data , 2018, Connect. Sci..

[10]  Keqin Li,et al.  Knowledge-maximized ensemble algorithm for different types of concept drift , 2018, Inf. Sci..

[11]  Yunwen Xu,et al.  Power plant performance modeling with concept drift , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[12]  João Gama,et al.  A survey on concept drift adaptation , 2014, ACM Comput. Surv..

[13]  Manfred K. Warmuth,et al.  The Weighted Majority Algorithm , 1994, Inf. Comput..

[14]  Vincenzo Loia,et al.  Drift-Aware Methodology for Anomaly Detection in Smart Grid , 2019, IEEE Access.

[15]  Geoff Holmes,et al.  Improving Adaptive Bagging Methods for Evolving Data Streams , 2009, ACML.

[16]  Javier Del Ser,et al.  DRED: An evolutionary diversity generation method for concept drift adaptation in online learning environments , 2017, Appl. Soft Comput..

[17]  Xin Yao,et al.  Online Ensemble Learning of Data Streams with Gradually Evolved Classes , 2016, IEEE Transactions on Knowledge and Data Engineering.

[18]  Bartosz Krawczyk,et al.  Online ensemble learning with abstaining classifiers for drifting and noisy data streams , 2017, Appl. Soft Comput..

[19]  Claudio De Stefano,et al.  Reliable writer identification in medieval manuscripts through page layout features: The "Avila" Bible case , 2018, Eng. Appl. Artif. Intell..

[20]  Khaled Ghédira,et al.  Discussion and review on evolving data streams and concept drift adapting , 2018, Evol. Syst..

[21]  João Gama,et al.  A new dynamic modeling framework for credit risk assessment , 2016, Expert Syst. Appl..

[22]  Roberto Souto Maior de Barros,et al.  An overview and comprehensive comparison of ensembles for concept drift , 2019, Inf. Fusion.

[23]  Roberto Souto Maior de Barros,et al.  RDDM: Reactive drift detection method , 2017, Expert Syst. Appl..

[24]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[25]  Xin Yao,et al.  DDD: A New Ensemble Approach for Dealing with Concept Drift , 2012, IEEE Transactions on Knowledge and Data Engineering.

[26]  Marcus A. Maloof,et al.  Dynamic Weighted Majority: An Ensemble Method for Drifting Concepts , 2007, J. Mach. Learn. Res..

[27]  Roberto Souto Maior de Barros,et al.  Speeding Up Recovery from Concept Drifts , 2014, ECML/PKDD.

[28]  Roberto Souto Maior de Barros,et al.  A large-scale comparison of concept drift detectors , 2018, Inf. Sci..

[29]  Geoff Hulten,et al.  Mining time-changing data streams , 2001, KDD '01.

[30]  João Gama,et al.  Learning with Drift Detection , 2004, SBIA.

[31]  Geoff Holmes,et al.  Leveraging Bagging for Evolving Data Streams , 2010, ECML/PKDD.

[32]  Der-Jiunn Deng,et al.  Concept Drift Detection and Adaption in Big Imbalance Industrial IoT Data Using an Ensemble Learning Method of Offline Classifiers , 2019, IEEE Access.

[33]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[34]  Roberto Souto Maior de Barros,et al.  A Boosting-like Online Learning Ensemble , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).