Multivariate LSTM-FCNs for Time Series Classification

Over the past decade, multivariate time series classification has received great attention. We propose transforming the existing univariate time series classification models, the Long Short Term Memory Fully Convolutional Network (LSTM-FCN) and Attention LSTM-FCN (ALSTM-FCN), into a multivariate time series classification model by augmenting the fully convolutional block with a squeeze-and-excitation block to further improve accuracy. Our proposed models outperform most state-of-the-art models while requiring minimum preprocessing. The proposed models work efficiently on various complex multivariate time series classification tasks such as activity recognition or action recognition. Furthermore, the proposed models are highly efficient at test time and small enough to deploy on memory constrained systems.

[1]  Seungjin Choi,et al.  Bayesian common spatial patterns for multi-subject EEG classification , 2014, Neural Networks.

[2]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.

[3]  David M. J. Tax,et al.  Multivariate Time-Series Classification Using the Hidden-Unit Logistic Model , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[4]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[5]  Marc Toussaint,et al.  Modelling motion primitives and their timing in biologically executed movements , 2007, NIPS.

[6]  Gary King,et al.  Logistic Regression in Rare Events Data , 2001, Political Analysis.

[7]  Sahin Albayrak,et al.  Pattern recognition and classification for multivariate time series , 2011, SensorKDD '11.

[8]  Razvan Pascanu,et al.  How to Construct Deep Recurrent Neural Networks , 2013, ICLR.

[9]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[10]  Pierre Geurts,et al.  Pattern Extraction for Time Series Classification , 2001, PKDD.

[11]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[12]  David Haussler,et al.  A Discriminative Framework for Detecting Remote Protein Homologies , 2000, J. Comput. Biol..

[13]  Mohammed Waleed Kadous,et al.  Temporal classification: extending the classification paradigm to multivariate time series , 2002 .

[14]  Minho Lee,et al.  Real-time human action classification using a dynamic neural model , 2015, Neural Networks.

[15]  Arnaud Doucet,et al.  Autoregressive Kernels For Time Series , 2011, 1101.0673.

[16]  Luís Nunes,et al.  Human Activity Recognition and Prediction , 2015 .

[17]  Ying Wu,et al.  Mining actionlet ensemble for action recognition with depth cameras , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Ulf Leser,et al.  Multivariate Time Series Classification with WEASEL+MUSE , 2017, ArXiv.

[19]  Vladimir Pavlovic,et al.  Time-series classification using mixed-state dynamic Bayesian networks , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[20]  Emile A. Hendriks,et al.  Action unit classification using active appearance models and conditional random fields , 2011, Cognitive Processing.

[21]  Laurens van der Maaten,et al.  Learning Discriminative Fisher Kernels , 2011, ICML.

[22]  Wenyu Zhang,et al.  Multivariate Time Series Classification Using Dynamic Time Warping Template Selection for Human Activity Recognition , 2015, 2015 IEEE Symposium Series on Computational Intelligence.

[23]  Tim Oates,et al.  Time series classification from scratch with deep neural networks: A strong baseline , 2016, 2017 International Joint Conference on Neural Networks (IJCNN).

[24]  Trevor Darrell,et al.  Hidden Conditional Random Fields , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Houshang Darabi,et al.  LSTM Fully Convolutional Networks for Time Series Classification , 2017, IEEE Access.

[26]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[27]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[28]  Yi Zheng,et al.  Time Series Classification Using Multi-Channels Deep Convolutional Neural Networks , 2014, WAIM.

[29]  Juan José Rodríguez Diez,et al.  Stacking for multivariate time series classification , 2015, Pattern Analysis and Applications.

[30]  Wanqing Li,et al.  Action recognition based on a bag of 3D points , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[31]  Junyou Shi,et al.  Complex Rotation Quantum Dynamic Neural Networks (CRQDNN) using Complex Quantum Neuron (CQN): Applications to time series prediction , 2015, Neural Networks.

[32]  Lars Schmidt-Thieme,et al.  Ultra-Fast Shapelets for Time Series Classification , 2015, ArXiv.

[33]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[34]  Brahim Chaib-draa,et al.  Parametric Exponential Linear Unit for Deep Convolutional Neural Networks , 2016, 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA).

[35]  Athanasios Kehagias,et al.  Predictive Modular Neural Networks for Time Series Classification , 1997, Neural Networks.

[36]  Z. Šidák.  Rectangular Confidence Regions for the Means of Multivariate Normal Distributions , 1967 .

[37]  Mustafa Gokce Baydogan,et al.  Autoregressive forests for multivariate time series modeling , 2018, Pattern Recognit..

[38]  Shan Liu,et al.  An effective multivariate time series classification approach using echo state network and adaptive differential evolution algorithm , 2016, Expert Syst. Appl..

[39]  Nacereddine Hammami,et al.  Improved tree model for arabic speech recognition , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[40]  Gang Sun,et al.  Squeeze-and-Excitation Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  Bülent Sankur,et al.  Probabilistic sequence clustering with spectral learning , 2014, Digit. Signal Process..

[42]  Alex Graves,et al.  Supervised Sequence Labelling with Recurrent Neural Networks , 2012, Studies in Computational Intelligence.

[43]  Houshang Darabi,et al.  Efficient Classification of Long Time Series by 3-D Dynamic Time Warping , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[44]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[45]  Gregory D. Hager,et al.  Temporal Convolutional Networks: A Unified Approach to Action Segmentation , 2016, ECCV Workshops.

[46]  George C. Runger,et al.  Time series representation and similarity based on local autopatterns , 2016, Data Mining and Knowledge Discovery.

[47]  Jian Pei,et al.  A brief survey on sequence classification , 2010, SKDD.

[48]  Carlo Vercellis,et al.  Combining discrete SVM and fixed cardinality warping distances for multivariate time series classification , 2010, Pattern Recognit..

[49]  George C. Runger,et al.  Learning a symbolic representation for multivariate time series classification , 2015, Data Mining and Knowledge Discovery.

[50]  R. Rodrigo,et al.  Faster human activity recognition with SVM , 2012, International Conference on Advances in ICT for Emerging Regions (ICTer2012).