A Novel Weight Adjustment Method for Handling Concept-Drift in Data Stream Classification

Evolving fuzzy rule-based systems are very powerful methods for online classification of data streams. In these systems, the classifier is updated by generating, removing and modifying fuzzy classification rules. One of the simplest but efficient algorithms of this type is evolving classifier (eClass) that generates fuzzy classification rules without any prior knowledge. However, this algorithm cannot cope properly with drift and shift in the concept of data. In order to improve the performance of this algorithm, in this paper, we propose a scheme that assigns a weight to each fuzzy rule and propose a new efficient online method to adjust the weight of fuzzy classification rules. Using the proposed rule-weighting algorithm, the classifier can quickly cope with drift and shift in the concept of data. Our algorithm is in fact the modified version of a batch mode rule-weight learning algorithm proposed in the past to be consistent with characteristics of data streams. We use some real life and some synthetic datasets to assess the performance of our algorithm in comparison with eClass and some other methods proposed in the past for handling data streams. The results of experiments show that our proposed algorithm performs significantly better than eClass and other methods proposed in the past for classification of data streams.

[1]  M. Zolghadri Jahromi,et al.  A cost sensitive learning algorithm for intrusion detection , 2010, 2010 18th Iranian Conference on Electrical Engineering.

[2]  Hisao Ishibuchi,et al.  Selecting fuzzy if-then rules for classification problems using genetic algorithms , 1995, IEEE Trans. Fuzzy Syst..

[3]  Ludmila I. Kuncheva,et al.  How good are fuzzy If-Then classifiers? , 2000, IEEE Trans. Syst. Man Cybern. Part B.

[4]  Albert Bifet,et al.  DATA STREAM MINING A Practical Approach , 2009 .

[5]  Robert Ivor John,et al.  A method of learning weighted similarity function to improve the performance of nearest neighbor , 2009, Inf. Sci..

[6]  Geoff Hulten,et al.  Mining time-changing data streams , 2001, KDD '01.

[7]  João Gama,et al.  Learning with Drift Detection , 2004, SBIA.

[8]  Hisao Ishibuchi,et al.  Rule weight specification in fuzzy rule-based classification systems , 2005, IEEE Transactions on Fuzzy Systems.

[9]  Plamen P. Angelov,et al.  Identification of evolving fuzzy rule-based models , 2002, IEEE Trans. Fuzzy Syst..

[10]  Plamen P. Angelov,et al.  An approach for fuzzy rule-base adaptation using on-line clustering , 2004, Int. J. Approx. Reason..

[11]  Plamen P. Angelov,et al.  Architectures for evolving fuzzy rule-based classifiers , 2007, 2007 IEEE International Conference on Systems, Man and Cybernetics.

[12]  Plamen Angelov,et al.  Evolving Rule-Based Models: A Tool For Design Of Flexible Adaptive Systems , 2002 .

[13]  Hisao Ishibuchi,et al.  Adaptive fuzzy rule-based classification systems , 1996, IEEE Trans. Fuzzy Syst..

[14]  Albert Bifet,et al.  Massive Online Analysis , 2009 .

[15]  Philip S. Yu,et al.  Mining concept-drifting data streams using ensemble classifiers , 2003, KDD '03.

[16]  Francisco Herrera,et al.  Ten years of genetic fuzzy systems: current framework and new trends , 2004, Fuzzy Sets Syst..

[17]  Mansoor Zolghadri Jahromi,et al.  A proposed method for learning rule weights in fuzzy rule-based classification systems , 2008, Fuzzy Sets Syst..

[18]  Seyed Mostafa Fakhrahmad,et al.  Constructing accurate fuzzy classifiers: A new adaptive method for rule-weight specification , 2008, Int. J. Knowl. Based Intell. Eng. Syst..

[19]  Mansoor Zolghadri Jahromi,et al.  A Novel Piecewise Linear Clustering Technique Based on Hyper Plane Adjustment , 2008, CSICC.

[20]  Abraham Kandel,et al.  Real-time data mining of non-stationary data streams from sensor networks , 2008, Inf. Fusion.

[21]  Charles Elkan KDD'99 Knowledge Discovery Contest , 2000, SIGKDD Explor..

[22]  Mykola Pechenizkiy,et al.  Handling Local Concept Drift with Dynamic Integration of Classifiers: Domain of Antibiotic Resistance in Nosocomial Infections , 2006, 19th IEEE Symposium on Computer-Based Medical Systems (CBMS'06).

[23]  Plamen P. Angelov,et al.  Handling drifts and shifts in on-line data streams with evolving fuzzy systems , 2011, Appl. Soft Comput..

[24]  Eghbal G. Mansoori,et al.  A weighting function for improving fuzzy classification systems performance , 2007, Fuzzy Sets Syst..

[25]  Salwani Abdullah,et al.  Great Deluge Algorithm for Rough Set Attribute Reduction , 2010, FGIT-DTA/BSBT.

[26]  D.P. Filev,et al.  An approach to online identification of Takagi-Sugeno fuzzy models , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[27]  Marcus A. Maloof,et al.  Dynamic Weighted Majority: An Ensemble Method for Drifting Concepts , 2007, J. Mach. Learn. Res..

[28]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[29]  Carlo Zaniolo,et al.  An Adaptive Nearest Neighbor Classification Algorithm for Data Streams , 2005, PKDD.

[30]  Kun Zhang,et al.  Forecasting Skewed Biased Stochastic Ozone Days: Analyses and Solutions , 2006, Sixth International Conference on Data Mining (ICDM'06).

[31]  Grigorios Tsoumakas,et al.  An adaptive personalized news dissemination system , 2009, Journal of Intelligent Information Systems.

[32]  Plamen P. Angelov,et al.  Evolving Fuzzy-Rule-Based Classifiers From Data Streams , 2008, IEEE Transactions on Fuzzy Systems.

[33]  Edwin Lughofer,et al.  An approach to model-based fault detection in industrial measurement systems with application to engine test benches , 2006 .

[34]  Frank Klawonn,et al.  Mathematical Analysis of Fuzzy Classifiers , 1997, IDA.

[35]  P. Angelov,et al.  A Method for Predicting Quality of the Crude Oil Distillation , 2006, 2006 International Symposium on Evolving Fuzzy Systems.