Learning Discrete-Time Markov Chains Under Concept Drift

Learning under concept drift is a novel and promising research area aiming at designing learning algorithms able to deal with nonstationary data-generating processes. In this research field, most of the literature focuses on learning nonstationary probabilistic frameworks, while some extensions about learning graphs and signals under concept drift exist. For the first time in the literature, this paper addresses the problem of learning discrete-time Markov chains (DTMCs) under concept drift. More specifically, following a hybrid active/passive approach, this paper introduces both a family of change-detection mechanisms (CDMs), differing in the required assumptions and performance, for detecting changes in DTMCs and an adaptive learning algorithm able to deal with DTMCs under concept drift. The effectiveness of both the proposed CDMs and the adaptive learning algorithm has been extensively tested on synthetically generated experiments and real data sets.

[1]  M. Bartlett,et al.  Weak ergodicity in non-homogeneous Markov chains , 1958, Mathematical Proceedings of the Cambridge Philosophical Society.

[2]  Cesare Alippi,et al.  Hierarchical Change-Detection Tests , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Cesare Alippi.  Intelligence for Embedded Systems: A Methodological Approach , 2014 .

[4]  Ricard Gavaldà,et al.  Learning from Time-Changing Data with Adaptive Windowing , 2007, SDM.

[5]  Cesare Alippi,et al.  Just-in-Time Adaptive Classifiers—Part II: Designing the Classifier , 2008, IEEE Transactions on Neural Networks.

[6]  Cesare Alippi,et al.  An Improved Hilbert-Huang Transform for Non-linear and Time-Variant Signals , 2018, Multidisciplinary Approaches to Neural Computing.

[7]  Petar M. Djuric,et al.  An MCMC sampling approach to estimation of nonstationary hidden Markov models , 2002, IEEE Trans. Signal Process..

[8]  Robi Polikar,et al.  Guest Editorial Learning in Nonstationary and Evolving Environments , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Hamid Beigy,et al.  New Drift Detection Method for Data Streams , 2011, ICAIS.

[10]  Nong Ye,et al.  A Markov Chain Model of Temporal Behavior for Anomaly Detection , 2000 .

[11]  Gregory Ditzler,et al.  Learning in Nonstationary Environments: A Survey , 2015, IEEE Computational Intelligence Magazine.

[12]  A. Bifet,et al.  Early Drift Detection Method , 2005 .

[13]  João Gama,et al.  Learning with Drift Detection , 2004, SBIA.

[14]  George V. Moustakides.  Quickest Detection of Abrupt Changes for a Class of Random Processes , 1998, IEEE Trans. Inf. Theory.

[15]  Dimitris K. Tasoulis,et al.  Exponentially weighted moving average charts for detecting concept drift , 2012, Pattern Recognit. Lett..

[16]  Takashi Omori,et al.  Detection of unusual human behavior in intelligent house , 2002, Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing.

[17]  Cesare Alippi,et al.  The (Not) Far-Away Path to Smart Cyber-Physical Systems: An Information-Centric Framework , 2017, Computer.

[18]  Cesare Alippi,et al.  Learning in Nonstationary Environments: A Hybrid Approach , 2017, ICAISC.

[19]  Peter Willett,et al.  Detection of hidden Markov model transient signals , 2000, IEEE Trans. Aerosp. Electron. Syst..

[20]  J. Wolfowitz,et al.  Optimum Character of the Sequential Probability Ratio Test , 1948 .

[21]  William Nick Street,et al.  A streaming ensemble algorithm (SEA) for large-scale classification , 2001, KDD '01.

[22]  M. S. Bartlett,et al.  The ergodic properties of non-homogeneous finite Markov chains , 1956, Mathematical Proceedings of the Cambridge Philosophical Society.

[23]  R. Klein,et al.  A Bayesian approach to modelling the natural history of a chronic condition from observations with intervention. , 1999, Statistics in medicine.

[24]  Marcus A. Maloof,et al.  Dynamic weighted majority: a new ensemble method for tracking concept drift , 2003, Third IEEE International Conference on Data Mining.

[25]  Gregory Ditzler,et al.  Hellinger distance based drift detection for nonstationary environments , 2011, 2011 IEEE Symposium on Computational Intelligence in Dynamic and Uncertain Environments (CIDUE).

[26]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[27]  João Gama,et al.  A survey on concept drift adaptation , 2014, ACM Comput. Surv..

[28]  Shai Ben-David,et al.  Detecting Change in Data Streams , 2004, VLDB.

[29]  Tze Leung Lai,et al.  Information Bounds and Quick Detection of Parameter Changes in Stochastic Systems , 1998, IEEE Trans. Inf. Theory.

[30]  Alexey Tsymbal,et al.  The problem of concept drift: definitions and related work , 2004 .

[31]  Lorenzo Livi,et al.  Concept Drift and Anomaly Detection in Graph Streams , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Gregory Ditzler,et al.  Adaptive Classifiers for Nonstationary Environments , 2015 .

[33]  Xufeng Niu,et al.  Detecting Shifts in Hurricane Rates Using a Markov Chain Monte Carlo Approach , 2004 .

[34]  Jin H. Kim,et al.  Nonstationary hidden Markov model , 1995, Signal Process..

[35]  E. Lehmann.  Testing Statistical Hypotheses , 1960 .

[36]  B. Craig,et al.  Estimation of the transition matrix of a discrete-time Markov chain. , 2002, Health economics.

[37]  Cesare Alippi,et al.  Just-In-Time Classifiers for Recurrent Concepts , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[38]  Chao Zhang,et al.  Generalization Bounds of ERM-Based Learning Processes for Continuous-Time Markov Chains , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[39]  T. W. Anderson,et al.  Statistical Inference about Markov Chains , 1957 .

[40]  Mubarak Shah,et al.  Video scene segmentation using Markov chain Monte Carlo , 2006, IEEE Transactions on Multimedia.

[41]  Robi Polikar,et al.  Incremental Learning of Concept Drift in Nonstationary Environments , 2011, IEEE Transactions on Neural Networks.