On Handling Missing Values in Data Stream Mining Algorithms Based on the Restricted Boltzmann Machine

This paper addresses the issue of data stream mining using the Restricted Boltzmann Machine (RBM). Recently, it was demonstrated that the RBM can be useful as a concept drift detector in data streams with time-changing probability density. In this paper, we consider another problem which often occurs in real-life data streams, i.e. incomplete data. We propose two modifications of the RBM learning algorithms to make them able to handle missing values. The first one inserts an additional procedure before the positive phase of the Contrastive Divergence. This procedure aims at inferring the missing values in the visible layer by performing a fixed number of Gibbs steps. The second modification introduces dimension-dependent sizes of minibatches in the stochastic gradient descent method. The proposed methods are verified experimentally, demonstrating their usability for concept drift detection in data streams with incomplete data.

[1]  Piotr Duda,et al.  New Splitting Criteria for Decision Trees in Stationary Data Streams , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Geoff Holmes,et al.  Active Learning With Drifting Streaming Data , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Piotr Duda,et al.  Concept Drift Detection in Streams of Labelled Data Using the Restricted Boltzmann Machine , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[4]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[5]  Piotr Duda,et al.  On applying the Restricted Boltzmann Machine to active concept drift detection , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[6]  Nando de Freitas,et al.  An Introduction to MCMC for Machine Learning , 2004, Machine Learning.

[7]  Vincent Lemaire,et al.  A Survey on Supervised Classification on Data Streams , 2014, eBISS.

[8]  Geoffrey E. Hinton.  A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[9]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[10]  Bartosz Krawczyk,et al.  Online ensemble learning with abstaining classifiers for drifting and noisy data streams , 2017, Appl. Soft Comput..

[11]  V. Susheela Devi,et al.  Parallel MCNN (pMCNN) with Application to Prototype Selection on Large and Streaming Data , 2017, J. Artif. Intell. Soft Comput. Res..

[12]  Francisco Herrera,et al.  A survey on data preprocessing for data stream mining: Current status and future directions , 2017, Neurocomputing.

[13]  Andrzej Cader,et al.  Resource-Aware Data Stream Mining Using the Restricted Boltzmann Machine , 2019, ICAISC.