Few-shot Learning for Time-series Forecasting

Time-series forecasting is important for many applications. Forecasting models are usually trained using time-series data in a specific target task. However, sufficient data in the target task might be unavailable, which leads to performance degradation. In this paper, we propose a few-shot learning method that forecasts a future value of a time-series in a target task given a few time-series in the target task. Our model is trained using time-series data in multiple training tasks that are different from target tasks. Our model uses a few time-series to build a forecasting function based on a recurrent neural network with an attention mechanism. With the attention mechanism, we can retrieve useful patterns in a small number of time-series for the current situation. Our model is trained by minimizing an expected test error of forecasting next timestep values. We demonstrate the effectiveness of the proposed method using 90 time-series datasets.

[1]  Bin Yu,et al.  Spatio-temporal Graph Convolutional Neural Network: A Deep Learning Framework for Traffic Forecasting , 2017 .

[2]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[3]  Nicolas Chapados,et al.  Meta-learning framework with applications to zero-shot time-series forecasting , 2020, AAAI.

[4]  Joshua B. Tenenbaum,et al.  The Variational Homoencoder: Learning to learn high capacity generative models from few examples , 2018, UAI.

[5]  Michael I. Jordan,et al.  Deep Transfer Learning with Joint Adaptation Networks , 2016, ICML.

[6]  Yoshua Bengio,et al.  Learning a synaptic learning rule , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[7]  Rob J. Hyndman,et al.  Meta‐learning how to forecast time series , 2023, Journal of Forecasting.

[8]  Athanasios Sfetsos,et al.  A comparison of various forecasting techniques applied to mean hourly wind speed time series , 2000 .

[9]  Jörg Bornschein,et al.  Variational Memory Addressing in Generative Models , 2017, NIPS.

[10]  Dmitry P. Vetrov,et al.  Few-shot Generative Modelling with Generative Matching Networks , 2018, AISTATS.

[11]  Daan Wierstra,et al.  One-Shot Generalization in Deep Generative Models , 2016, ICML.

[12]  Francis Eng Hock Tay,et al.  Support vector machine with adaptive parameters in financial time series forecasting , 2003, IEEE Trans. Neural Networks.

[13]  Tahseen Ahmed Jilani,et al.  Multivariate High Order Fuzzy Time Series Forecasting for Car Road Accidents , 2008 .

[14]  Patrick Nguyen,et al.  Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis , 2018, NeurIPS.

[15]  Feng Liu,et al.  Meta Learning with Relational Information for Short Sequences , 2019, NeurIPS.

[16]  Yee Whye Teh,et al.  Attentive Neural Processes , 2019, ICLR.

[17]  J. David Fuller,et al.  Backpropagation in Hydrological Time Series Forecasting , 1994 .

[18]  Sergey Levine,et al.  Probabilistic Model-Agnostic Meta-Learning , 2018, NeurIPS.

[19]  Finale Doshi-Velez,et al.  Robust and Efficient Transfer Learning with Hidden Parameter Markov Decision Processes , 2017, AAAI.

[20]  Ratnesh Sharma,et al.  Energy Predictive Models with Limited Data using Transfer Learning , 2019, e-Energy.

[21]  Amos J. Storkey,et al.  Towards a Neural Statistician , 2016, ICLR.

[22]  Wenshuo Wang,et al.  Recurrent Attentive Neural Process for Sequential Data , 2019, ArXiv.

[23]  Richard S. Zemel,et al.  Prototypical Networks for Few-shot Learning , 2017, NIPS.

[24]  J. Yosinski,et al.  Time-series Extreme Event Forecasting with Neural Networks at Uber , 2017 .

[25]  Razvan Pascanu,et al.  Meta-Learning with Latent Embedding Optimization , 2018, ICLR.

[26]  Cyrus Shahabi,et al.  Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting , 2017, ICLR.

[27]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[28]  Hang Li,et al.  Meta-SGD: Learning to Learn Quickly for Few Shot Learning , 2017, ArXiv.

[29]  Lovekesh Vig,et al.  Meta-Learning for Few-Shot Time Series Classification , 2019, COMAD/CODS.

[30]  Bogdan Gabrys,et al.  Meta-learning for time series forecasting and forecast combination , 2010, Neurocomputing.

[31]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[32]  Nicolas Chapados,et al.  N-BEATS: Neural basis expansion analysis for interpretable time series forecasting , 2019, ICLR.

[33]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[34]  Bogdan Gabrys,et al.  Cross-domain Meta-learning for Time-series Forecasting , 2018, KES.

[35]  AssaadMohammad,et al.  A new boosting algorithm for improved time-series forecasting with recurrent neural networks , 2008 .

[36]  Marcin Andrychowicz,et al.  Learning to learn by gradient descent by gradient descent , 2016, NIPS.

[37]  Brenden M. Lake,et al.  Compositional generalization through meta sequence-to-sequence learning , 2019, NeurIPS.

[38]  Kyoung-jae Kim,et al.  Financial time series forecasting using support vector machines , 2003, Neurocomputing.

[39]  E. Michael Azoff,et al.  Neural Network Time Series: Forecasting of Financial Markets , 1994 .

[40]  Steve B. Jiang,et al.  Nonlinear Systems Identification Using Deep Dynamic Neural Networks , 2016, ArXiv.

[41]  Chao Yang,et al.  A Survey on Deep Transfer Learning , 2018, ICANN.

[42]  Lu Liu,et al.  Few-shot Time-series Classification with Dual Interpretability , 2019 .

[43]  Yasuhiro Fujiwara,et al.  Transfer Anomaly Detection by Inferring Latent Domain Representations , 2019, NeurIPS.

[44]  Fan Zhang,et al.  A review on time series forecasting techniques for building energy consumption , 2017 .

[45]  Ying Wei,et al.  Hierarchically Structured Meta-learning , 2019, ICML.

[46]  Hugo Larochelle,et al.  Optimization as a Model for Few-Shot Learning , 2016, ICLR.

[47]  Yoshua Bengio,et al.  Bayesian Model-Agnostic Meta-Learning , 2018, NeurIPS.

[48]  Hubert Cardot,et al.  A new boosting algorithm for improved time-series forecasting with recurrent neural networks , 2008, Inf. Fusion.

[49]  Teresa Bernarda Ludermir,et al.  Meta-learning approaches to selecting time series models , 2004, Neurocomputing.

[50]  Thomas Paine,et al.  Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions , 2017, ICLR.

[51]  Christian Osendorfer,et al.  Recurrent Neural Processes , 2019, ArXiv.

[52]  Yee Whye Teh,et al.  Conditional Neural Processes , 2018, ICML.

[53]  Miriam A. M. Capretz,et al.  Transfer learning with seasonal and trend adjustment for cross-building energy forecasting , 2018 .