Multidimensional continuous time Bayesian network classifiers

The multidimensional classification of multivariate time series deals with the assignment of multiple classes to time‐ordered data described by a set of feature variables. Although this challenging task has received almost no attention in the literature, it is present in a wide variety of domains, such as medicine, finance or industry. The complexity of this problem lies in two nontrivial tasks, the learning with multivariate time series in continuous time and the simultaneous classification of multiple class variables that may show dependencies between them. These can be addressed with different strategies, but most of them may involve a difficult preprocessing of the data, high space and classification complexity or ignoring useful interclass dependencies. Additionally, no attention has been given to the development of new multidimensional classifiers of time series based on probabilistic graphical models, even though transparent models can facilitate further understanding of the domain. In this paper, a novel probabilistic graphical model is proposed, which is able to classify a discrete multivariate temporal sequence into multiple class variables while modeling their dependencies. This model extends continuous time Bayesian networks to the multidimensional classification problem, which are able to explicitly represent the behavior of time series that evolve over continuous time. Different methods for the learning of the parameters and structure of the model are presented, and numerical experiments on synthetic and real‐world data show encouraging results in terms of performance and learning time with respect to independent classifiers, the current alternative approach under the continuous time Bayesian network paradigm.

[1]  Concha Bielza,et al.  New insights into the classification and nomenclature of cortical GABAergic interneurons , 2013, Nature Reviews Neuroscience.

[2]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[3]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Genetic Algorithms: A Performance Analysis of Control Parameters , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Yannis Manolopoulos,et al.  Feature-based classification of time-series data , 2001 .

[5]  Daphne Koller,et al.  Learning Continuous Time Bayesian Networks , 2002, UAI.

[6]  Aurélien Mazurie,et al.  Gene networks inference using dynamic Bayesian networks , 2003, ECCB.

[7]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[8]  Gregory F. Cooper,et al.  Scoring Bayesian networks of mixed variables , 2018, International Journal of Data Science and Analytics.

[9]  Concha Bielza,et al.  Multi-dimensional Bayesian network classifiers: A survey , 2020, Artificial Intelligence Review.

[10]  Jiebo Luo,et al.  A Bayesian network-based framework for semantic image understanding , 2005, Pattern Recognit..

[11]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[12]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[13]  Eamonn J. Keogh,et al.  A Complexity-Invariant Distance Measure for Time Series , 2011, SDM.

[14]  Jason Weston,et al.  A kernel method for multi-labelled classification , 2001, NIPS.

[15]  Olufemi A. Omitaomu,et al.  Weighted dynamic time warping for time series classification , 2011, Pattern Recognit..

[16]  Concha Bielza,et al.  Discrete Bayesian Network Classifiers , 2014, ACM Comput. Surv..

[17]  David Heckerman,et al.  A Tutorial on Learning with Bayesian Networks , 1999, Innovations in Bayesian Networks.

[18]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[19]  Iñaki Inza,et al.  Supervised pre-processing approaches in multiple class variables classification for fish recruitment forecasting , 2013, Environ. Model. Softw..

[20]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[21]  Amanda Clare,et al.  Knowledge Discovery in Multi-label Phenotype Data , 2001, PKDD.

[22]  Fabio Stella,et al.  Continuous time Bayesian network classifiers , 2012, J. Biomed. Informatics.

[23]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[24]  Ivor W. Tsang,et al.  Survey on Multi-Output Learning , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Min-Ling Zhang,et al.  Multi-dimensional classification via kNN feature augmentation , 2020, Pattern Recognit..

[26]  Yuval Shahar,et al.  Classification-driven temporal discretization of multivariate time series , 2014, Data Mining and Knowledge Discovery.

[27]  David Maxwell Chickering,et al.  Learning Bayesian networks: The combination of knowledge and statistical data , 1995, Mach. Learn..

[28]  Concha Bielza,et al.  Multi-dimensional classification with Bayesian networks , 2011, Int. J. Approx. Reason..

[29]  Keiji Kanazawa,et al.  A model for reasoning about persistence and causation , 1989 .

[30]  Donald J. Berndt,et al.  Using Dynamic Time Warping to Find Patterns in Time Series , 1994, KDD Workshop.

[31]  Linda C. van der Gaag,et al.  Inference and Learning in Multi-dimensional Bayesian Network Classifiers , 2007, ECSQARU.

[32]  Daphne Koller,et al.  Continuous Time Bayesian Networks , 2012, UAI.

[33]  R. Bouckaert Bayesian belief networks : from construction to inference , 1995 .

[34]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[35]  Fabio Stella,et al.  Learning continuous time Bayesian network classifiers , 2014, Int. J. Approx. Reason..

[36]  Fabio Stella,et al.  Learning Continuous Time Bayesian Networks in Non-stationary Domains , 2016, J. Artif. Intell. Res..

[37]  Geoff Holmes,et al.  Classifier chains for multi-label classification , 2009, Machine Learning.

[38]  Fabio Stella,et al.  Making Continuous Time Bayesian Networks More Flexible , 2018, PGM.

[39]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[40]  Hamilton E. Link,et al.  Discrete dynamic Bayesian network analysis of fMRI data , 2009, Human brain mapping.

[41]  Li Yang,et al.  On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice , 2020, Neurocomputing.

[42]  Nick S. Jones,et al.  Highly Comparative Feature-Based Time-Series Classification , 2014, IEEE Transactions on Knowledge and Data Engineering.

[43]  Friedhelm Schwenker,et al.  Classification of bioacoustic time series based on the combination of global and local decisions , 2004, Pattern Recognit..

[44]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[45]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[46]  Charles Elkan,et al.  Learning to Diagnose with LSTM Recurrent Neural Networks , 2015, ICLR.

[47]  Concha Bielza,et al.  A Bayesian network model for surface roughness prediction in the machining process , 2008, Int. J. Syst. Sci..

[48]  H. Akaike A new look at the statistical model identification , 1974 .

[49]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[50]  Jasper Snoek,et al.  Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[51]  Geoffrey Zweig,et al.  Speech Recognition with Dynamic Bayesian Networks , 1998, AAAI/IAAI.

[52]  Fabio Stella,et al.  Constraint-Based Learning for Continuous-Time Bayesian Networks , 2020, PGM.

[53]  Linda C. van der Gaag,et al.  Multi-dimensional Bayesian Network Classifiers , 2006, Probabilistic Graphical Models.

[54]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[55]  Min-Ling Zhang,et al.  Multi-Dimensional Classification via kNN Feature Augmentation , 2019, AAAI.