Stochastic discretized learning-based weak estimation: a novel estimation method for non-stationary environments

The task of designing estimators that are able to track time-varying distributions has found promising applications in many real-life problems.Existing approaches resort to sliding windows that track changes by discarding old observations. In this paper, we report a novel estimator referred to as the Stochastic Discretized Weak Estimator (SDWE), that is based on the principles of discretized Learning Automata (LA). In brief, the estimator is able to estimate the parameters of a time varying binomial distribution using finite memory. The estimator tracks changes in the distribution by operating a controlled random walk in a discretized probability space. The steps of the estimator are discretized so that the updates are done in jumps, and thus the convergence speed is increased. Further, the state transitions are both state-dependent and randomized. As far as we know, such a scheme is both novel and pioneering. The results which have first been proven for binomial distributions have subsequently been extended for the multinomial case, using which they can be applied to any univariate distribution using a histogram-based scheme. The most outstanding and pioneering contribution of our work is that of achieving multinomial estimation without relying on a set of binomial estimators, and where the underlying strategy is truly randomized. Interestingly, the estimator possesses a low computational complexity that is independent of the number of parameters of the multinomial distribution. The generalization of these results for other distributions has also been alluded to. The paper briefly reports conclusive experimental results that prove the ability of the SDWE to cope with non-stationary environments with high adaptation and accuracy. HighlightsWe present finite-memory estimation techniques for non-stationary environments.These new techniques use the principles of discretized Learning Automata.The results have been tested for many time-varying problems and applications.

[1]  Edith Cohen,et al.  Maintaining time-decaying stream aggregates , 2003, J. Algorithms.

[2]  B. John Oommen,et al.  Solving Stochastic Nonlinear Resource Allocation Problems Using a Hierarchy of Twofold Resource Allocation Automata , 2010, IEEE Transactions on Computers.

[3]  Frank Spitzer.  Optimal Stopping Rules (A. N. Shiryayev) , 1981 .

[4]  P. S. Sastry,et al.  Varieties of learning automata: an overview , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[5]  Ivan Koychev,et al.  Tracking Drifting Concepts by Time Window Optimisation , 2005, SGAI Conf..

[6]  ThathacharM. A.L.,et al.  Varieties of learning automata , 2002 .

[7]  Ralf Klinkenberg,et al.  Learning drifting concepts: Example selection vs. example weighting , 2004, Intell. Data Anal..

[8]  Michèle Basseville,et al.  Detection of abrupt changes: theory and application , 1993 .

[9]  Anis Yazidi,et al.  A User-Centric Approach for Personalized Service Provisioning in Pervasive Environments , 2011, Wirel. Pers. Commun..

[10]  OommenB. John,et al.  Fault-tolerant routing in adversarial mobile ad hoc networks , 2010 .

[11]  B. John Oommen,et al.  Anomaly Detection in Dynamic Systems Using Weak Estimators , 2011, TOIT.

[12]  Abdelhamid Bouchachia,et al.  Fuzzy classification in dynamic environments , 2011, Soft Comput..

[13]  Anis Yazidi,et al.  An Adaptive Approach to Learning the Preferences of Users in a Social Network Using Weak Estimators , 2012, J. Inf. Process. Syst..

[14]  Mykola Pechenizkiy,et al.  Dynamic integration of classifiers for handling concept drift , 2008, Inf. Fusion.

[15]  B. John Oommen,et al.  Continuous and discretized pursuit learning schemes: various algorithms and their comparison , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[16]  Hongjoong Kim,et al.  A novel approach to detection of intrusions in computer networks via adaptive sequential and batch-sequential change-point detection methods , 2006, IEEE Transactions on Signal Processing.

[17]  Ivan Koychev,et al.  Gradual Forgetting for Adaptation to Concept Drift , 2000 .

[18]  D. Kendall.  Stochastic Processes and Population Growth , 1949 .

[19]  B. John Oommen,et al.  The Use of Weak estimators to Achieve Language Detection and Tracking in Multilingual Documents , 2013, Int. J. Pattern Recognit. Artif. Intell..

[20]  Dimitris K. Tasoulis,et al.  Exponentially weighted moving average charts for detecting concept drift , 2012, Pattern Recognit. Lett..

[21]  B. John Oommen,et al.  Learning Automata-Based Solutions to the Nonlinear Fractional Knapsack Problem With Applications to Optimal Resource Allocation , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[22]  Alʹbert Nikolaevich Shiri︠a︡ev,et al.  Optimal stopping rules , 1977 .

[23]  B. John Oommen,et al.  A Novel Strategy for Solving the Stochastic Point Location Problem Using a Hierarchical Searching Scheme , 2014, IEEE Transactions on Cybernetics.

[24]  B. John Oommen,et al.  Fault-tolerant routing in adversarial mobile ad hoc networks: an efficient route estimation scheme for non-stationary environments , 2010, Telecommun. Syst..

[25]  João Gama,et al.  Change Detection in Learning Histograms from Data Streams , 2007, EPIA Workshops.

[26]  Kumpati S. Narendra,et al.  Learning automata - an introduction , 1989 .

[27]  Roshani Ade,et al.  Incremental Learning From Unbalanced Data with Concept Class, Concept Drift and Missing Features : A Review , 2014 .

[28]  Koen Vanhoof,et al.  Research Challenges in Ubiquitous Knowledge Discovery , 2008, Next Generation of Data Mining.

[29]  Gerhard Widmer,et al.  Learning in the presence of concept drift and hidden contexts , 2004, Machine Learning.

[30]  Anton Dries,et al.  Adaptive concept drift detection , 2009, SDM.

[31]  B. John Oommen,et al.  Generalized pursuit learning schemes: new families of continuous and discretized learning automata , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[32]  M. Thathachar,et al.  Networks of Learning Automata: Techniques for Online Stochastic Optimization , 2003 .

[33]  B. John Oommen,et al.  Absorbing and Ergodic Discretized Two-Action Learning Automata , 1986, IEEE Trans. Syst. Man Cybern..

[34]  B.J. Oommen,et al.  Stochastic Automata-Based Estimators for Adaptively Compressing Files With Nonstationary Distributions , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[35]  João Gama,et al.  A survey on concept drift adaptation , 2014, ACM Comput. Surv..

[36]  Ivan Koychev,et al.  Adaptation to Drifting User's Interests , 2000 .

[37]  B. John Oommen,et al.  Stochastic learning-based weak estimation of multinomial random variables and its applications to pattern recognition in non-stationary environments , 2006, Pattern Recognit..

[38]  B. John Oommen,et al.  The asymptotic optimality of discretized linear reward-inaction learning automata , 1984, IEEE Transactions on Systems, Man, and Cybernetics.

[39]  B. John Oommen,et al.  Discretized estimator learning automata , 1992, IEEE Trans. Syst. Man Cybern..

[40]  Geoff Hulten,et al.  Mining high-speed data streams , 2000, KDD '00.

[41]  S. Venkatasubramanian,et al.  An Information-Theoretic Approach to Detecting Changes in Multi-Dimensional Data Streams , 2006 .