Autonomous Learning Multimodel Systems From Data Streams

In this paper, an approach to autonomous learning of a multimodel system from streaming data, named ALMMo, is proposed. The proposed approach is generic and can easily be applied also to probabilistic or other types of local models forming multimodel systems. It is fully data driven and its structure is decided by the nonparametric data clouds extracted from the empirically observed data without making any prior assumptions concerning data distribution and other data properties. All metaparameters of the proposed system are obtained directly from the data and can be updated recursively, which improves memory and calculation efficiencies of the proposed algorithm. The structural evolution mechanism and online data cloud quality monitoring mechanism of the ALMMo system largely enhance the ability of handling shifts and/or drifts in the streaming data pattern. Numerical examples of the use of ALMMo system for streaming data analytics, classification, and prediction are presented as a proof of the proposed concept.

[1]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[2]  D.P. Filev,et al.  An approach to online identification of Takagi-Sugeno fuzzy models , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[3]  Xiaowei Gu,et al.  MICE: Multi-Layer Multi-Model Images Classifier Ensemble , 2017, 2017 3rd IEEE International Conference on Cybernetics (CYBCON).

[4]  Plamen P. Angelov,et al.  Simpl_eClass: Simplified potential-free evolving fuzzy rule-based classifiers , 2011, 2011 IEEE International Conference on Systems, Man, and Cybernetics.

[5]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[6]  Yuni Xia,et al.  StreamFitter: A Real Time Linear Regression Analysis System for Continuous Data Streams , 2011, DASFAA.

[7]  Plamen P. Angelov,et al.  Evolving Fuzzy-Rule-Based Classifiers From Data Streams , 2008, IEEE Transactions on Fuzzy Systems.

[8]  V. Bianco,et al.  Electricity consumption forecasting in Italy using linear regression models , 2009 .

[9]  K. Narendra,et al.  Adaptive Control Using Collective Information Obtained from Multiple Models , 2011 .

[10]  Paramasivan Saratchandran,et al.  Sequential Adaptive Fuzzy Inference System (SAFIS) for nonlinear system identification and prediction , 2006, Fuzzy Sets Syst..

[11]  Lotfi A. Zadeh,et al.  Outline of a New Approach to the Analysis of Complex Systems and Decision Processes , 1973, IEEE Trans. Syst. Man Cybern..

[12]  Jyh-Horng Chou,et al.  Design of Optimal Controllers for Takagi–Sugeno Fuzzy-Model-Based Systems , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[13]  Krzysztof Zaremba,et al.  Self-Organising Maps for Classification with Metropolis-Hastings Algorithm for Supervision , 2012, ICONIP.

[14]  J. G. Saw,et al.  Chebyshev Inequality With Estimated Mean and Variance , 1984 .

[15]  Plamen P. Angelov,et al.  A Generalized Methodology for Data Analysis , 2018, IEEE Transactions on Cybernetics.

[16]  Plamen P. Angelov,et al.  Evolving Single- And Multi-Model Fuzzy Classifiers with FLEXFIS-Class , 2007, 2007 IEEE International Fuzzy Systems Conference.

[17]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[18]  Plamen P. Angelov,et al.  PANFIS: A Novel Incremental Learning Machine , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[19]  D. M. Titterington,et al.  Variational approximations in Bayesian model selection for finite mixture distributions , 2007, Comput. Stat. Data Anal..

[20]  L X Wang,et al.  Fuzzy basis functions, universal approximation, and orthogonal least-squares learning , 1992, IEEE Trans. Neural Networks.

[21]  Plamen P. Angelov,et al.  Empirical Fuzzy Sets , 2018, Int. J. Intell. Syst..

[22]  Plamen P. Angelov,et al.  Fuzzily Connected Multimodel Systems Evolving Autonomously From Data Streams , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[23]  Hiok Chai Quek,et al.  A BCM Theory of Meta-Plasticity for Online Self-Reorganizing Fuzzy-Associative Learning , 2010, IEEE Transactions on Neural Networks.

[24]  Kumpati S. Narendra,et al.  Adaptation and learning using multiple models, switching, and tuning , 1995 .

[25]  Samuel Kotz,et al.  Probability Integrals of the Multivariate t Distribution , 2008 .

[26]  Alex Graves,et al.  Supervised Sequence Labelling with Recurrent Neural Networks , 2012, Studies in Computational Intelligence.

[27]  Plamen P. Angelov,et al.  A new type of simplified fuzzy rule-based system , 2012, Int. J. Gen. Syst..

[28]  S. Ya. Shatskikh.  Multivariate cauchy distributions as locally gaussian distributions , 1996 .

[29]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[30]  Xiaowei Gu,et al.  Empirical Data Analytics , 2017, Int. J. Intell. Syst..

[31]  Ebrahim H. Mamdani,et al.  An Experiment in Linguistic Synthesis with a Fuzzy Logic Controller , 1999, Int. J. Hum. Comput. Stud..

[32]  F. Gers,et al.  Long short-term memory in recurrent neural networks , 2001 .

[33]  Nikola K. Kasabov,et al.  Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[34]  Kumpati S. Narendra,et al.  Adaptive control using multiple models , 1997, IEEE Trans. Autom. Control..

[35]  Plamen Angelov,et al.  Autonomous Learning Systems: From Data Streams to Knowledge in Real-time , 2013 .

[36]  Nikola K. Kasabov,et al.  DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction , 2002, IEEE Trans. Fuzzy Syst..

[37]  Franziska Hoffmann,et al.  Spatial Tessellations Concepts And Applications Of Voronoi Diagrams , 2016 .

[38]  Frank Klawonn,et al.  Effects of drift and noise on the optimal sliding window size for data stream regression models , 2017 .