Probabilistic Forecasting of Sensory Data With Generative Adversarial Networks – ForGAN

Time series forecasting is one of the challenging problems for humankind. The traditional forecasting methods using mean regression models have severe shortcomings in reflecting real-world fluctuations. While new probabilistic methods rush to rescue, they fight with technical difficulties like quantile crossing or selecting a prior distribution. To meld the different strengths of these fields while avoiding their weaknesses, as well as, to push the boundary of the state-of-the-art, we introduce ForGAN – one step ahead probabilistic forecasting with generative adversarial networks. ForGAN utilizes the power of the conditional generative adversarial network to learn the data generating distribution and compute probabilistic forecasts from it. We argue how to evaluate ForGAN in opposition to regression methods. To investigate probabilistic forecasting of ForGAN, we create a new dataset and demonstrate our method abilities on it. This dataset will be made publicly available for comparison. Furthermore, we test ForGAN on two publicly available datasets, namely Mackey-Glass dataset and Internet traffic dataset (A5M), where the impressive performance of ForGAN demonstrate its high capability in forecasting future values.

[1]  Mrinalini Shah,et al.  Fuzzy based trend mapping and forecasting for time series data , 2012, Expert Syst. Appl..

[2]  J. Tennyson In the wake of chaos. Unpredictable order in dynamical systems , 1995 .

[3]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[4]  H. Pomares,et al.  A heuristic method for parameter selection in LS-SVM: Application to time series prediction , 2011 .

[5]  Jyh-Shing Roger Jang,et al.  SVSGAN: Singing Voice Separation Via Generative Adversarial Network , 2017, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  Viktor K. Prasanna,et al.  Generative Adversarial Network for Synthetic Time Series Data Generation in Smart Grids , 2018, 2018 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm).

[7]  Prabhat,et al.  ExtremeWeather: A large-scale climate dataset for semi-supervised detection, localization, and understanding of extreme weather events , 2016, NIPS.

[8]  Tom Minka,et al.  Expectation Propagation for approximate Bayesian inference , 2001, UAI.

[9]  Torsten Hothorn,et al.  Conditional transformation models , 2012, 1201.5786.

[10]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[11]  Gerassimos A. Papadopoulos,et al.  OPERATIONAL EARTHQUAKE FORECASTING. State of Knowledge and Guidelines for Utilization , 2011 .

[12]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[13]  Ta-Chung Chu,et al.  Application of fuzzy multiple attribute decision making on company analysis for stock selection , 1996, Soft Computing in Intelligent Systems and Information Processing. Proceedings of the 1996 Asian Fuzzy Systems Symposium.

[14]  Ingoo Han,et al.  Genetic algorithms approach to feature discretization in artificial neural networks for the prediction of stock price index , 2000 .

[15]  Yang Gao,et al.  Voice Impersonation Using Generative Adversarial Networks , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Francesco Ravazzolo,et al.  Real-Time Inflation Forecasting in a Changing World , 2009 .

[17]  Chris Donahue,et al.  Adversarial Audio Synthesis , 2018, ICLR.

[18]  Lalit M. Patnaik,et al.  Genetic algorithms: a survey , 1994, Computer.

[19]  Çagdas Hakan Aladag,et al.  Fuzzy time series forecasting with a novel hybrid approach combining fuzzy c-means and neural networks , 2013, Expert Syst. Appl..

[20]  Alan Ritter,et al.  Adversarial Learning for Neural Dialogue Generation , 2017, EMNLP.

[21]  Daniel Kirschen,et al.  Model-Free Renewable Scenario Generation Using Generative Adversarial Networks , 2017, IEEE Transactions on Power Systems.

[22]  Radford M. Neal Probabilistic Inference Using Markov Chain Monte Carlo Methods , 2011 .

[23]  Roman Krzysztofowicz,et al.  The case for probabilistic forecasting in hydrology , 2001 .

[24]  Bo Wang,et al.  Machine Learning for Integrating Data in Biology and Medicine: Principles, Practice, and Opportunities , 2018, Inf. Fusion.

[25]  Zheng-Hua Tan,et al.  Conditional Generative Adversarial Networks for Speech Enhancement and Noise-Robust Speaker Verification , 2017, INTERSPEECH.

[26]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[27]  Paul H. C. Eilers,et al.  Simultaneous estimation of quantile curves using quantile sheets , 2013 .

[28]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[29]  Lovekesh Vig,et al.  Long Short Term Memory Networks for Anomaly Detection in Time Series , 2015, ESANN.

[30]  Yarin Gal,et al.  Uncertainty in Deep Learning , 2016 .

[31]  Zoubin Ghahramani,et al.  Probabilistic machine learning and artificial intelligence , 2015, Nature.

[32]  K. Knuth Optimal Data-Based Binning for Histograms , 2006, physics/0605197.

[33]  Adrian E. Raftery,et al.  Probabilistic projections of HIV prevalence using Bayesian melding. , 2007, 0709.0421.

[34]  Chris Donahue,et al.  Synthesizing Audio with Generative Adversarial Networks , 2018, ArXiv.

[35]  Adrian E. Raftery,et al.  Weather Forecasting with Ensemble Methods , 2005, Science.

[36]  Georg Dorffner,et al.  Neural Networks for Time Series Processing , 1996 .

[37]  M. Genton,et al.  Short‐Term Wind Speed Forecasting for Power System Operations , 2012 .

[38]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[39]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine Learning.

[40]  Steve B. Jiang,et al.  Nonlinear Systems Identification Using Deep Dynamic Neural Networks , 2016, ArXiv.

[41]  Freda Kemp,et al.  An Introduction to Sequential Monte Carlo Methods , 2003 .

[42]  Bo Wu,et al.  A Novel Stock Forecasting Model based on Fuzzy Time Series and Genetic Algorithm , 2013, ICCS.

[43]  Lantao Yu,et al.  SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient , 2016, AAAI.

[44]  Zhe Gan,et al.  Generating Text via Adversarial Training , 2016 .

[45]  Michael E Phelps,et al.  Systems Biology and New Technologies Enable Predictive and Preventative Medicine , 2004, Science.

[46]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[47]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[48]  Thomas Kneib,et al.  Geoadditive expectile regression , 2012, Comput. Stat. Data Anal..

[49]  Florian M. Hollenbach,et al.  Ensemble Predictions of the 2012 US Presidential Election , 2012, PS: Political Science & Politics.

[50]  W. Ronny Huang,et al.  Accurate, Data-Efficient Learning from Noisy, Choice-Based Labels for Inherent Risk Scoring , 2018, ArXiv.

[51]  A. Raftery,et al.  Using Bayesian Model Averaging to Calibrate Forecast Ensembles , 2005 .

[52]  Andrew Y. Ng,et al.  Predicting Inpatient Discharge Prioritization With Electronic Health Records , 2018, ArXiv.

[53]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[54]  David J. Spiegelhalter,et al.  Improved probabilistic prediction of healthcare performance indicators using bidirectional smoothing models , 2012 .

[55]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[56]  Allan Timmermann,et al.  Density forecasting in economics and finance , 2000 .

[57]  Xing Yan,et al.  A comparison between SVM and LSSVM in mid-term electricity market clearing price forecasting , 2013, 2013 26th IEEE Canadian Conference on Electrical and Computer Engineering (CCECE).

[58]  Huaiyu Zhu On Information and Sufficiency , 1997 .

[59]  T. Kneib Beyond mean regression , 2013 .

[60]  W. Newey,et al.  Asymmetric Least Squares Estimation and Testing , 1987 .

[61]  Florian Pappenberger,et al.  Ensemble flood forecasting: a review. , 2009 .

[62]  C. Sparrow The Lorenz Equations: Bifurcations, Chaos, and Strange Attractors , 1982 .

[63]  Nando de Freitas,et al.  An Introduction to Sequential Monte Carlo Methods , 2001, Sequential Monte Carlo Methods in Practice.

[64]  M. Collins Ensembles and probabilities: a new era in the prediction of climate change , 2007, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[65]  Tim N. Palmer,et al.  The economic value of ensemble forecasts as a tool for risk assessment: From days to decades , 2002 .

[66]  Tim Palmer Towards the probabilistic Earth‐system simulator: a vision for the future of climate and weather prediction , 2012 .

[67]  Yisong Yue,et al.  PhaseLink: A Deep Learning Approach to Seismic Phase Association , 2018, Journal of Geophysical Research: Solid Earth.

[68]  Cheng-Lung Huang,et al.  A GA-based feature selection and parameters optimizationfor support vector machines , 2006, Expert Syst. Appl..

[69]  Pedro Sousa,et al.  Multi‐scale Internet traffic forecasting using neural networks and time series methods , 2010, Expert Syst. J. Knowl. Eng..

[70]  Olof Mogren,et al.  C-RNN-GAN: Continuous recurrent neural networks with adversarial training , 2016, ArXiv.

[71]  See-Kiong Ng,et al.  Anomaly Detection with Generative Adversarial Networks for Multivariate Time Series , 2018, ArXiv.

[72]  Qiang Song,et al.  A new fuzzy time-series model of fuzzy number observations , 1995 .

[73]  Thomas Janssoone,et al.  Machine Learning on Electronic Health Records: Models and Features Usages to predict Medication Non-Adherence , 2018, ArXiv.

[74]  R. Koenker,et al.  Regression Quantiles , 2007 .

[75]  Patricia Melin,et al.  A Competitive Modular Neural Network for Long-Term Time Series Forecasting , 2017, Nature-Inspired Design of Hybrid Intelligent Systems.

[76]  Bernhard Schölkopf,et al.  Feature selection for support vector machines by means of genetic algorithm , 2003, Proceedings. 15th IEEE International Conference on Tools with Artificial Intelligence.

[77]  José Neves,et al.  Genetic and Evolutionary Algorithms for Time Series Forecasting , 2001, IEA/AIE.

[78]  Erol Egrioglu,et al.  A modified genetic algorithm for forecasting fuzzy time series , 2014, Applied Intelligence.

[79]  Hui Peng,et al.  A hybrid algorithm to optimize RBF network architecture and parameters for nonlinear time series prediction , 2012 .

[80]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[81]  Xing Yan,et al.  Mid-term electricity market clearing price forecasting using multiple least squares support vector machines , 2014 .

[82]  Andrew Y. Ng,et al.  Improving palliative care with deep learning , 2017, 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[83]  Gunnar Rätsch,et al.  Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs , 2017, ArXiv.

[84]  T. Papanastasiou,et al.  Viscous Fluid Flow , 1999 .

[85]  Naomi S. Altman,et al.  Quantile regression , 2019, Nature Methods.

[86]  Nan Li,et al.  Bayesian probabilistic population projections for all countries , 2012, Proceedings of the National Academy of Sciences.

[87]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[88]  Bernhard Sick,et al.  A Multi-Scheme Ensemble Using Coopetitive Soft-Gating With Application to Power Forecasting for Renewable Energy Generation , 2018, 1803.06344.

[89]  R. A. Wentzell,et al.  Hydrodynamic and Hydromagnetic Stability. By S. CHANDRASEKHAR. Clarendon Press: Oxford University Press, 1961. 652 pp. £5. 5s. , 1962, Journal of Fluid Mechanics.

[90]  Neil D. Lawrence,et al.  Deep Gaussian Processes , 2012, AISTATS.

[91]  Trevor Darrell,et al.  Adversarial Feature Learning , 2016, ICLR.

[92]  Holger Dette,et al.  Non‐crossing non‐parametric estimates of quantile curves , 2008 .

[93]  Chris Donahue,et al.  Exploring Speech Enhancement with Generative Adversarial Networks for Robust Speech Recognition , 2017, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[94]  S. Mostafa Mousavi,et al.  CRED: A Deep Residual Network of Convolutional and Recurrent Units for Earthquake Signal Detection , 2018, Scientific Reports.

[95]  Qiang Zhang,et al.  Benchmarking Deep Sequential Models on Volatility Predictions for Financial Time Series , 2018, ArXiv.

[96]  Marco Aurélio Stelmar Netto,et al.  DeepDownscale: A Deep Learning Strategy for High-Resolution Weather Forecast , 2018, 2018 IEEE 14th International Conference on e-Science (e-Science).

[97]  Pierre Pinson,et al.  Wind Energy: Forecasting Challenges for Its Operational Management , 2013, 1312.6471.

[98]  Ami Wiesel,et al.  ML for Flood Forecasting at Scale , 2019, ArXiv.

[99]  Lior Wolf,et al.  Language Generation with Recurrent Generative Adversarial Networks without Pre-training , 2017, ArXiv.

[100]  Çagdas Hakan Aladag,et al.  A new time invariant fuzzy time series forecasting method based on particle swarm optimization , 2012, Appl. Soft Comput..

[101]  Hubert Cardot,et al.  A new boosting algorithm for improved time-series forecasting with recurrent neural networks , 2008, Inf. Fusion.