Benchmarking Deep Learning for Time Series: Challenges and Directions

Deep learning for time series is an emerging area with close ties to industry, yet under represented in performance benchmarks for machine learning systems. In this paper, we present a landscape of deep learning applications applied to time series, and discuss the challenges and directions towards building a robust performance benchmark of deep learning workloads for time series data.

[1]  Xiaohui Xie,et al.  Co-Occurrence Feature Learning for Skeleton Based Action Recognition Using Regularized Deep LSTM Networks , 2016, AAAI.

[2]  Valentin Flunkert,et al.  DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks , 2017, International Journal of Forecasting.

[3]  Torsten Hoefler,et al.  A Modular Benchmarking Infrastructure for High-Performance and Reproducible Deep Learning , 2019, 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[4]  Kunle Olukotun,et al.  DAWNBench : An End-to-End Deep Learning Benchmark and Competition , 2017 .

[5]  Alexis Giguere,et al.  Natural Time, Nowcasting and the Physics of Earthquakes: Estimation of Seismic Risk to Global Megacities , 2017, Pure and Applied Geophysics.

[6]  Qiang Zhang,et al.  Benchmarking Deep Sequential Models on Volatility Predictions for Financial Time Series , 2018, ArXiv.

[7]  Takuya Shintate,et al.  Trend Prediction Classification for High Frequency Bitcoin Time Series with Deep Learning , 2019, Journal of Risk and Financial Management.

[8]  Yan Liu,et al.  Hierarchical Deep Generative Models for Multi-Rate Multivariate Time Series , 2018, ICML.

[9]  Cyrus Shahabi,et al.  Graph Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting , 2017, ArXiv.

[10]  Tailai Wen,et al.  Time Series Anomaly Detection Using Convolutional Neural Networks and Transfer Learning , 2019, ArXiv.

[11]  Zhixin Sun,et al.  A Survey of Techniques for Mobile Service Encrypted Traffic Classification Using Deep Learning , 2019, IEEE Access.

[12]  Jonathan D. Cryer,et al.  Time Series Analysis , 1986 .

[13]  F. Viégas,et al.  Deep learning of aftershock patterns following large earthquakes , 2018, Nature.

[14]  Daniel S. Berman,et al.  A Survey of Deep Learning Methods for Cyber Security , 2019, Inf..

[15]  Tiago H. Falk,et al.  Deep learning-based electroencephalography analysis: a systematic review , 2019, Journal of neural engineering.

[16]  Christoph Bergmeir,et al.  Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions , 2019, ArXiv.

[17]  Xinyu Li,et al.  A Survey of Deep Learning-Based Human Activity Recognition in Radar , 2019, Remote. Sens..

[18]  David Patterson,et al.  MLPerf Training Benchmark , 2019, MLSys.

[19]  Ying Wah Teh,et al.  Time-series clustering - A decade review , 2015, Inf. Syst..

[20]  Ruofeng Wen,et al.  Deep Generative Quantile-Copula Models for Probabilistic Forecasting , 2019, ArXiv.

[21]  Carole-Jean Wu,et al.  MLPerf Inference Benchmark , 2020, 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA).

[22]  Paul A. Johnson,et al.  A Silent Build-up in Seismic Energy Precedes Slow Slip Failure in the Cascadia Subduction Zone , 2019, 1909.06787.

[23]  Alexandros Iosifidis,et al.  Benchmark Dataset for Mid-Price Prediction of Limit Order Book data , 2017, ArXiv.

[24]  Amar Phanishayee,et al.  Benchmarking and Analyzing Deep Neural Network Training , 2018, 2018 IEEE International Symposium on Workload Characterization (IISWC).

[25]  Lior Rokach,et al.  SherLock vs Moriarty: A Smartphone Dataset for Cybersecurity Research , 2016, AISec@CCS.

[26]  Xin Liu,et al.  Deep Learning for Encrypted Traffic Classification: An Overview , 2018, IEEE Communications Magazine.

[27]  Zhanxing Zhu,et al.  ST-UNet: A Spatio-Temporal U-Network for Graph-structured Time Series Modeling , 2019, ArXiv.

[28]  Ivan Poupyrev,et al.  Interacting with Soli: Exploring Fine-Grained Dynamic Gesture Recognition in the Radio-Frequency Spectrum , 2016, UIST.

[29]  Kaung Khin,et al.  Fast ES-RNN: A GPU Implementation of the ES-RNN Algorithm , 2019, ArXiv.

[30]  Alexander Krauss,et al.  Machine learning-based run-time anomaly detection in software systems: An industrial evaluation , 2018, 2018 IEEE Workshop on Machine Learning Techniques for Software Quality Evaluation (MaLTeSQuE).

[31]  Paul Lukowicz,et al.  Collecting complex activity datasets in highly rich networked sensor environments , 2010, 2010 Seventh International Conference on Networked Sensing Systems (INSS).

[32]  Subhrajit Roy,et al.  ChronoNet: A Deep Recurrent Neural Network for Abnormal EEG Identification , 2018, AIME.

[33]  J. Yosinski,et al.  Time-series Extreme Event Forecasting with Neural Networks at Uber , 2017 .

[34]  Germain Forestier,et al.  Deep learning for time series classification: a review , 2018, Data Mining and Knowledge Discovery.

[35]  N. Birbaumer,et al.  BCI2000: a general-purpose brain-computer interface (BCI) system , 2004, IEEE Transactions on Biomedical Engineering.

[36]  Gautier Marti,et al.  Autoregressive Convolutional Neural Networks for Asynchronous Time Series , 2017, ICML.

[37]  Julian Kates-Harbeck,et al.  Predicting disruptive instabilities in controlled fusion plasmas through deep learning , 2019, Nature.

[38]  Pavel Filonov,et al.  Multivariate Industrial Time Series with Cyber-Attack Simulation: Fault Detection Using an LSTM-based Predictive Data Model , 2016, ArXiv.

[39]  Stefan Zohren,et al.  DeepLOB: Deep Convolutional Neural Networks for Limit Order Books , 2018, IEEE Transactions on Signal Processing.

[40]  Ivan Poupyrev,et al.  Soli , 2016, ACM Trans. Graph..

[41]  Nashat T. AL-Jallad,et al.  Hybrid Forecasting Scheme for Financial Time-Series Data using Neural Network and Statistical Methods , 2017 .

[42]  Gu-Yeon Wei,et al.  Benchmarking TPU, GPU, and CPU Platforms for Deep Learning , 2019, ArXiv.

[43]  John Cristian Borges Gamboa,et al.  Deep Learning for Time-Series Analysis , 2017, ArXiv.

[44]  Masoumeh Haghpanahi,et al.  Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network , 2019, Nature Medicine.

[45]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[46]  Cyrus Shahabi,et al.  Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting , 2017, ICLR.

[47]  Nikolay Laptev,et al.  Deep and Confident Prediction for Time Series at Uber , 2017, 2017 IEEE International Conference on Data Mining Workshops (ICDMW).

[48]  Ricardo Chavarriaga,et al.  The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition , 2013, Pattern Recognit. Lett..

[49]  Ali A. Ghorbani,et al.  Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization , 2018, ICISSP.

[50]  Daniel Roggen,et al.  Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition , 2016, Sensors.

[51]  Jinjun Xiong,et al.  Across-Stack Profiling and Characterization of Machine Learning Models on GPUs , 2019, ArXiv.

[52]  Mohammed Yeasin,et al.  Learning Representations from EEG with Deep Recurrent-Convolutional Neural Networks , 2015, ICLR.

[53]  Jie Chen,et al.  Anti-Money Laundering in Bitcoin: Experimenting with Graph Convolutional Networks for Financial Forensics , 2019, ArXiv.