Handling Time Changing Data with Adaptive Very Fast Decision Rules

Data streams are usually characterized by changes in the underlying distribution generating data. Therefore algorithms designed to work with data streams should be able to detect changes and quickly adapt the decision model. Rules are one of the most interpretable and flexible models for data mining prediction tasks. In this paper we present the Adaptive Very Fast Decision Rules (AVFDR), an on-line, any-time and one-pass algorithm for learning decision rules in the context of time changing data. AVFDR can learn ordered and unordered rule sets. It is able to adapt the decision model via incremental induction and specialization of rules. Detecting local drifts takes advantage of the modularity of rule sets. In AVFDR, each individual rule monitors the evolution of performance metrics to detect concept drift. AVFDR prunes rules that detect drift. This explicit change detection mechanism provides useful information about the dynamics of the process generating data, faster adaption to changes and generates compact rule sets. The experimental evaluation shows this method is able to learn fast and compact rule sets from evolving streams in comparison to alternative methods.

[1]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[2]  Ryszard S. Michalski,et al.  Incremental learning with partial instance memory , 2002, Artif. Intell..

[3]  Ralf Klinkenberg,et al.  Learning drifting concepts: Example selection vs. example weighting , 2004, Intell. Data Anal..

[4]  Philip S. Yu,et al.  Mining concept-drifting data streams using ensemble classifiers , 2003, KDD '03.

[5]  Jean-François Boulicaut,et al.  Advances in Intelligent Data Analysis VIII, 8th International Symposium on Intelligent Data Analysis, IDA 2009, Lyon, France, August 31 - September 2, 2009. Proceedings , 2009, IDA.

[6]  Pedro M. Domingos.  Unifying Instance-Based and Rule-Based Induction , 1996, Machine Learning.

[7]  Maliha S. Nash,et al.  Handbook of Parametric and Nonparametric Statistical Procedures , 2001, Technometrics.

[8]  William Nick Street,et al.  A streaming ensemble algorithm (SEA) for large-scale classification , 2001, KDD '01.

[9]  Geoff Holmes,et al.  MOA: Massive Online Analysis , 2010, J. Mach. Learn. Res..

[10]  Jesús S. Aguilar-Ruiz,et al.  Knowledge discovery from data streams , 2009, Intell. Data Anal..

[11]  João Gama,et al.  Accurate decision trees for mining high-speed data streams , 2003, KDD '03.

[12]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[13]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[14]  Geoff Hulten,et al.  Mining high-speed data streams , 2000, KDD '00.

[15]  João Gama,et al.  Very Fast Decision Rules for multi-class problems , 2012, SAC '12.

[16]  Raghu Ramakrishnan,et al.  Proceedings : KDD 2000 : the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 20-23, 2000, Boston, MA, USA , 2000 .

[17]  Alessandra Russo,et al.  Advances in Artificial Intelligence – SBIA 2004 , 2004, Lecture Notes in Computer Science.

[18]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[19]  Saso Dzeroski,et al.  Learning model trees from evolving data streams , 2010, Data Mining and Knowledge Discovery.

[20]  Eugene L. Grant,et al.  Statistical Quality Control , 1946 .

[21]  Sholom M. Weiss,et al.  Predictive data mining - a practical guide , 1997 .

[22]  João Gama,et al.  Decision trees for mining data streams , 2006, Intell. Data Anal..

[23]  Ronald L. Rivest,et al.  Learning decision lists , 2004, Machine Learning.

[24]  GamaJoão,et al.  Decision trees for mining data streams , 2006 .

[25]  João Gama,et al.  Learning Decision Rules from Data Streams , 2011, IJCAI.

[26]  Ricard Gavaldà,et al.  Adaptive Learning from Evolving Data Streams , 2009, IDA.

[27]  Peter Clark,et al.  The CN2 Induction Algorithm , 1989, Machine Learning.

[28]  Peter Clark,et al.  Rule Induction with CN2: Some Recent Improvements , 1991, EWSL.

[29]  Thorsten Meinl,et al.  KNIME - the Konstanz information miner: version 2.0 and beyond , 2009, SKDD.

[30]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[31]  J. C. Schlimmer,et al.  Incremental learning from noisy data , 2004, Machine Learning.

[32]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[33]  Jesús S. Aguilar-Ruiz,et al.  Incremental Rule Learning and Border Examples Selection from Numerical Data Streams , 2005, J. Univers. Comput. Sci..

[34]  A. Bifet,et al.  Early Drift Detection Method , 2005 .

[35]  Gerhard Widmer,et al.  Learning in the Presence of Concept Drift and Hidden Contexts , 1996, Machine Learning.

[36]  João Gama,et al.  Issues in evaluation of stream learning algorithms , 2009, KDD.

[37]  Geoff Holmes,et al.  New ensemble methods for evolving data streams , 2009, KDD.

[38]  Vipin Kumar,et al.  Chapman & Hall/CRC Data Mining and Knowledge Discovery Series , 2008 .

[39]  D. Hinkley.  Inference about the change-point from cumulative sum tests , 1971 .

[40]  Geoff Hulten,et al.  Mining time-changing data streams , 2001, KDD '01.

[41]  João Gama,et al.  Learning with Drift Detection , 2004, SBIA.