Efficient data stream classification via probabilistic adaptive windows

In the context of a data stream, a classifier must be able to learn from a theoretically-infinite stream of examples using limited time and memory, while being able to predict at any point. Many methods deal with this problem by basing their model on a window of examples. We introduce a probabilistic adaptive window (PAW) for data-stream learning, which improves this windowing technique with a mechanism to include older examples as well as the most recent ones, thus maintaining information on past concept drifts while being able to adapt quickly to new ones. We exemplify PAW with lazy learning methods in two variations: one to handle concept drift explicitly, and the other to add classifier diversity using an ensemble. Along with the standard measures of accuracy and time and memory use, we compare classifiers against state-of-the-art classifiers from the data-stream literature.

[1]  Geoff Holmes,et al.  Leveraging Bagging for Evolving Data Streams , 2010, ECML/PKDD.

[2]  M. Harries.  SPLICE-2 Comparative Evaluation: Electricity Pricing , 1999 .

[3]  Geoff Holmes,et al.  Batch-Incremental versus Instance-Incremental Learning in Dynamic and Evolving Data , 2012, IDA.

[4]  Geoff Hulten,et al.  Mining high-speed data streams , 2000, KDD '00.

[5]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[6]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[7]  Ricard Gavaldà,et al.  Adaptive Learning from Evolving Data Streams , 2009, IDA.

[8]  Ricard Gavaldà,et al.  Learning from Time-Changing Data with Adaptive Windowing , 2007, SDM.

[9]  Geoff Holmes,et al.  Fast Perceptron Decision Tree Learning from Evolving Data Streams , 2010, PAKDD.

[10]  Eyke Hüllermeier,et al.  IBLStreams: a system for instance-based classification and regression on data streams , 2012, Evolving Systems.

[11]  Philippe Flajolet,et al.  Approximate counting: A detailed analysis , 1985, BIT.

[12]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[13]  Li Guo,et al.  Enabling Fast Lazy Learning for Data Streams , 2011, 2011 IEEE 11th International Conference on Data Mining.

[14]  João Gama,et al.  Accurate decision trees for mining high-speed data streams , 2003, KDD '03.

[15]  Robert H. Morris,et al.  Counting large numbers of events in small registers , 1978, CACM.

[16]  Carlo Zaniolo,et al.  An Adaptive Nearest Neighbor Classification Algorithm for Data Streams , 2005, PKDD.

[17]  William Nick Street,et al.  A streaming ensemble algorithm (SEA) for large-scale classification , 2001, KDD '01.

[18]  Geoff Holmes,et al.  MOA: Massive Online Analysis , 2010, J. Mach. Learn. Res..

[19]  Stuart J. Russell,et al.  Experimental comparisons of online and batch versions of bagging and boosting , 2001, KDD '01.

[20]  Geoff Hulten,et al.  Mining time-changing data streams , 2001, KDD '01.

[21]  João Gama,et al.  Learning with Drift Detection , 2004, SBIA.

[22]  Geoff Holmes,et al.  New ensemble methods for evolving data streams , 2009, KDD.

[23]  Grigorios Tsoumakas,et al.  Dealing with Concept Drift and Class Imbalance in Multi-Label Stream Classification , 2011, IJCAI.