Multi-period classification: learning sequent classes from temporal domains

As the majority of real-world decisions change over time, extending traditional classifiers to deal with the problem of classifying an attribute of interest across different time periods becomes increasingly important. Tackling this problem, referred to as multi-period classification, is critical to answer real-world tasks, such as the prediction of upcoming healthcare needs or administrative planning tasks. In this context, although existing research provides principles for learning single labels from complex data domains, less attention has been given to the problem of learning sequences of classes (symbolic time series). This work motivates the need for multi-period classifiers, and proposes a method, cluster-based multi-period classification (CMPC), that preserves local dependencies across the periods under classification. Evaluation against real-world datasets provides evidence of the relevance of multi-period classifiers, and shows the superior performance of the CMPC method against peer methods adapted from long-term prediction for multi-period tasks with a high number of periods.

[1]  J. H. Ward Hierarchical Grouping to Optimize an Objective Function , 1963 .

[2]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[3]  Hans-Peter Kriegel,et al.  Density‐based clustering , 2011, WIREs Data Mining Knowl. Discov..

[4]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[5]  Li Wei,et al.  Fast time series classification using numerosity reduction , 2006, ICML.

[6]  Paul S. Bradley,et al.  Clustering very large databases using EM mixture models , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[7]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[8]  Francisco Azuaje,et al.  Integrative Data Analysis for Biomarker Discovery , 2010 .

[9]  Shuzlina Abdul Rahman,et al.  A Review on Protein Sequence Clustering Research , 2008 .

[10]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[11]  Francisco Azuaje,et al.  Bioinformatics and biomarker discovery : "omic" data analysis for personalised medicine , 2010 .

[12]  Fabian Mörchen,et al.  Time Series Knowledge Mining , 2006 .

[13]  John F. Roddick,et al.  A Survey of Temporal Knowledge Discovery Paradigms and Methods , 2002, IEEE Trans. Knowl. Data Eng..

[14]  Zhi-Hua Zhou,et al.  A k-nearest neighbor based algorithm for multi-label classification , 2005, 2005 IEEE International Conference on Granular Computing.

[15]  M. A. McClure,et al.  Hidden Markov models of biological primary sequence information. , 1994, Proceedings of the National Academy of Sciences of the United States of America.

[16]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[17]  Samy Bengio,et al.  Use of modular architectures for time series prediction , 1996, Neural Processing Letters.

[18]  Roger Lee Computer and Information Science 2012 , 2012 .

[19]  Gianluca Bontempi,et al.  Conditionally dependent strategies for multiple-step-ahead prediction in local learning , 2011 .

[20]  Hui Ding,et al.  Querying and mining of time series data: experimental comparison of representations and distance measures , 2008, Proc. VLDB Endow..

[21]  Pierre Geurts,et al.  Pattern Extraction for Time Series Classification , 2001, PKDD.

[22]  Amaury Lendasse,et al.  Methodology for long-term prediction of time series , 2007, Neurocomputing.

[23]  Antti Sorjamaa,et al.  Multiple-output modeling for multi-step-ahead time series forecasting , 2010, Neurocomputing.

[24]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[25]  Richard J. Povinelli,et al.  Time series classification using Gaussian mixture models of reconstructed phase spaces , 2004, IEEE Transactions on Knowledge and Data Engineering.

[26]  Amaury Lendasse,et al.  Long-term prediction of time series by combining direct and MIMO strategies , 2009, 2009 International Joint Conference on Neural Networks.

[27]  Johan A. K. Suykens,et al.  Lazy learning for iterated time-series prediction , 1998 .

[28]  Amaury Lendasse,et al.  Direct and Recursive Prediction of Time Series Using Mutual Information Selection , 2005, IWANN.

[29]  Cynthia Rudin,et al.  Sequential event prediction , 2013, Machine Learning.

[30]  Amine Bermak,et al.  Gaussian process for nonstationary time series prediction , 2004, Comput. Stat. Data Anal..

[31]  Cláudia Antunes,et al.  On the Need of New Approaches for the Novel Problem of Long-Term Prediction over Multi-dimensional Data , 2012 .

[32]  Hongxing He,et al.  Feature Selection for Temporal Health Records , 2001, PAKDD.

[33]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[34]  F. Mörchen Time series feature extraction for data mining using DWT and DFT , 2003 .

[35]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[36]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[37]  Padhraic Smyth,et al.  Modeling of multivariate time series using hidden markov models , 2005 .

[38]  Amir F. Atiya,et al.  A review and comparison of strategies for multi-step ahead time series forecasting based on the NN5 forecasting competition , 2011, Expert Syst. Appl..

[39]  Antonio Restivo,et al.  Distance measures for biological sequences: Some recent approaches , 2008, Int. J. Approx. Reason..

[40]  Alan J Lockett and Risto Miikkulainen Temporal Convolution Machines for Sequence Learning , 2009 .

[41]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[42]  Pirjo Moen,et al.  Attribute, Event Sequence, and Event Type Similarity Notions for Data Mining , 2000 .

[43]  Cláudia Antunes,et al.  Learning Predictive Models from Integrated Healthcare Data: Extending Pattern-Based and Generative Models to Capture Temporal and Cross-Attribute Dependencies , 2014, 2014 47th Hawaii International Conference on System Sciences.

[44]  Amaury Lendasse,et al.  Time series prediction using DirRec strategy , 2006, ESANN.

[45]  Eamonn J. Keogh,et al.  A Complexity-Invariant Distance Measure for Time Series , 2011, SDM.

[46]  L. K. Hansen,et al.  On Clustering fMRI Time Series , 1999, NeuroImage.

[47]  Vincent S. Tseng,et al.  Effective temporal data classification by integrating sequential pattern mining and probabilistic induction , 2009, Expert Syst. Appl..

[48]  Padhraic Smyth,et al.  Visualization of navigation patterns on a Web site using model-based clustering , 2000, KDD '00.

[49]  Alex Graves,et al.  Supervised Sequence Labelling with Recurrent Neural Networks , 2012, Studies in Computational Intelligence.

[50]  Wee Keong Ng,et al.  Closed motifs for streaming time series classification , 2013, Knowledge and Information Systems.