Gated Recurrent Neural Networks Empirical Utilization for Time Series Classification

Hybrid LSTM-Fully Convolutional Networks(LSTM-FCN) for time series classification has produced state-of-the-art classification results on univariate time series. This paper shows empirically that replacing the LSTM with a gated recurrent unit (GRU) to create a hybrid GRU fully convolutional network (GRU-FCN) can offer even better performance on many time series datasets. This resulted GRU-FCN model outperforms the state-of-the-art classification performance in many univariate time series datasets. In addition, since the GRU uses a simpler architecture than the LSTM, it has a simpler hardware implementation and fewer arithmetic components compared to the LSTM-based models.

[1]  Jason Lines,et al.  Time series classification with ensembles of elastic distance measures , 2015, Data Mining and Knowledge Discovery.

[2]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[3]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[4]  Eamonn J. Keogh,et al.  Exact indexing of dynamic time warping , 2002, Knowledge and Information Systems.

[5]  T. Pohlert.  The Pairwise Multiple Comparison of Mean Ranks Package (PMCMR) , 2016 .

[6]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[7]  Olufemi A. Omitaomu,et al.  Weighted dynamic time warping for time series classification , 2011, Pattern Recognit..

[8]  Yixin Chen,et al.  Multi-Scale Convolutional Neural Networks for Time Series Classification , 2016, ArXiv.

[9]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[10]  Mustafa Gul,et al.  Statistical pattern recognition for Structural Health Monitoring using time series modeling: Theory and experimental verifications , 2009 .

[11]  Jonathan D. Cryer,et al.  Time Series Analysis , 1986 .

[12]  J. Rotton,et al.  Air pollution, weather, and violent crimes: concomitant time-series analysis of archival data. , 1985, Journal of personality and social psychology.

[13]  Tim Oates,et al.  Time series classification from scratch with deep neural networks: A strong baseline , 2016, 2017 International Joint Conference on Neural Networks (IJCNN).

[14]  Chih-Hsiang Ho,et al.  Time Series Analysis for Predicting the Occurrences of Large Scale Earthquakes , 2012 .

[15]  Patrick Schäfer.  The BOSS is concerned with time series classification in the presence of noise , 2014, Data Mining and Knowledge Discovery.

[16]  Houshang Darabi,et al.  LSTM Fully Convolutional Networks for Time Series Classification , 2017, IEEE Access.

[17]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[18]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[19]  George C. Runger,et al.  A Bag-of-Features Framework to Classify Time Series , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Hoon Sohn,et al.  Damage diagnosis using time series analysis of vibration signals , 2001 .

[21]  Jason Lines,et al.  Time-Series Classification with COTE: The Collective of Transformation-Based Ensembles , 2015, IEEE Transactions on Knowledge and Data Engineering.

[22]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.