Concept Drift Detection in Streams of Labelled Data Using the Restricted Boltzmann Machine

In this paper, the method of concept drift detection in time-varying data stream mining is considered. The Restricted Boltzmann Machine (RBM) is proposed to be applied as a drift detector. The RBMs which are able to learn joint probability distributions of attribute values and their classes were taken into account. Properly learned they contain a compressed information about the underlying data distribution. The RBM learned on a part of the data stream can be used to determine possible changes in the data stream probability distribution. Two evaluation measures are applied as indicators of possible sudden or gradual changes: the reconstruction error and the free energy. In experiments conducted on synthetic datasets, both measures proved to be well suited for the task of concept drift detection.

[1]  João Gama,et al.  Issues in evaluation of stream learning algorithms , 2009, KDD.

[2]  Piotr Duda,et al.  A New Method for Data Stream Mining Based on the Misclassification Error , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Piotr Duda,et al.  Convergent Time-Varying Regression Models for Data Streams: Tracking Concept Drift by the Recursive Parzen-Based Generalized Regression Neural Networks , 2017, Int. J. Neural Syst..

[4]  Yoshua Bengio,et al.  Classification using discriminative restricted Boltzmann machines , 2008, ICML '08.

[5]  Francisco Herrera,et al.  A survey on data preprocessing for data stream mining: Current status and future directions , 2017, Neurocomputing.

[6]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[7]  Piotr Duda,et al.  New Splitting Criteria for Decision Trees in Stationary Data Streams , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[8]  V. Susheela Devi,et al.  Parallel MCNN (pMCNN) with Application to Prototype Selection on Large and Streaming Data , 2017, J. Artif. Intell. Soft Comput. Res..

[9]  João Gama,et al.  Learning with Drift Detection , 2004, SBIA.

[10]  Piotr Duda,et al.  Knowledge discovery in data streams with the orthogonal series-based generalized regression neural networks , 2017, Inf. Sci..

[11]  A. Bifet,et al.  Early Drift Detection Method , 2005 .

[12]  Jerzy Stefanowski,et al.  Reacting to Different Types of Concept Drift: The Accuracy Updated Ensemble Algorithm , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[13]  Ernestina Menasalvas Ruiz,et al.  Mining Recurring Concepts in a Dynamic Feature Space , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Haibo He,et al.  Incremental Learning From Stream Data , 2011, IEEE Transactions on Neural Networks.

[15]  Nicolas Le Roux,et al.  Representational Power of Restricted Boltzmann Machines and Deep Belief Networks , 2008, Neural Computation.

[16]  Geoff Hulten,et al.  Mining high-speed data streams , 2000, KDD '00.

[17]  Ricard Gavaldà,et al.  Learning from Time-Changing Data with Adaptive Windowing , 2007, SDM.

[18]  Geoff Holmes,et al.  Active Learning With Drifting Streaming Data , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Albert Bifet,et al.  Adaptive Stream Mining: Pattern Learning and Mining from Evolving Data Streams , 2010, Frontiers in Artificial Intelligence and Applications.

[20]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[21]  Piotr Duda,et al.  On applying the Restricted Boltzmann Machine to active concept drift detection , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).