Time series forecasting of petroleum production using deep LSTM recurrent networks

Abstract Time series forecasting (TSF) is the task of predicting future values of a given sequence using historical data. Recently, this task has attracted the attention of researchers in the area of machine learning to address the limitations of traditional forecasting methods, which are time-consuming and full of complexity. With the increasing availability of extensive amounts of historical data along with the need of performing accurate production forecasting, particularly a powerful forecasting technique infers the stochastic dependency between past and future values is highly needed. In this paper, we propose a deep learning approach capable to address the limitations of traditional forecasting approaches and show accurate predictions. The proposed approach is a deep long-short term memory (DLSTM) architecture, as an extension of the traditional recurrent neural network. Genetic algorithm is applied in order to optimally configure DLSTM’s optimum architecture. For evaluation purpose, two case studies from the petroleum industry domain are carried out using the production data of two actual oilfields. Toward a fair evaluation, the performance of the proposed approach is compared with several standard methods, either statistical or soft computing. Using different measurement criteria, the empirical results show that the proposed DLSTM model outperforms other standard approaches.

[1]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Donald Poskitt,et al.  The selection and use of linear and bilinear time series models , 1986 .

[3]  Roar Nybø Fault detection and other time series opportunities in the petroleum industry , 2010, Neurocomputing.

[4]  Chih-Ling Tsai,et al.  MODEL SELECTION FOR MULTIVARIATE REGRESSION IN SMALL SAMPLES , 1994 .

[5]  Jonathan D. Cryer,et al.  Time Series Analysis , 1986 .

[6]  Razvan Pascanu,et al.  How to Construct Deep Recurrent Neural Networks , 2013, ICLR.

[7]  Madan M. Gupta,et al.  An innovative neural forecast of cumulative oil production from a petroleum reservoir employing higher-order neural networks (HONNs) , 2013 .

[8]  Paul E. Utgoff,et al.  Many-Layered Learning , 2002, Neural Computation.

[9]  Amir F. Atiya,et al.  A review and comparison of strategies for multi-step ahead time series forecasting based on the NN5 forecasting competition , 2011, Expert Syst. Appl..

[10]  EunSu Lee,et al.  Forecasting Oil Production in North Dakota Using the Seasonal Autoregressive Integrated Moving Average (S-ARIMA) , 2015 .

[11]  Skipper Seabold,et al.  Statsmodels: Econometric and Statistical Modeling with Python , 2010, SciPy.

[12]  Igor N. Aizenberg,et al.  Multilayer Neural Network with Multi-Valued Neurons in time series forecasting of oil production , 2014, Neurocomputing.

[13]  José M. Molina López,et al.  Anomaly Detection Based on Sensor Data in Petroleum Industry Applications , 2015, Sensors.

[14]  Xin Ma,et al.  Predicting the oil production using the novel multivariate nonlinear model based on Arps decline model and kernel method , 2016, Neural Computing and Applications.

[15]  Min Xie,et al.  The use of ARIMA models for reliability forecasting and analysis , 1998 .

[16]  Amir H. Mohammadi,et al.  Decline curve based models for predicting natural gas well performance , 2017 .

[17]  Ridha Gharbi,et al.  An introduction to artificial intelligence applications in petroleum exploration and production , 2005 .

[18]  Amy Loutfi,et al.  A review of unsupervised feature learning and deep learning for time-series modeling , 2014, Pattern Recognit. Lett..

[19]  Rob J Hyndman,et al.  Another look at measures of forecast accuracy , 2006 .

[20]  R. Engle Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation , 1982 .

[21]  Benjamin Schrauwen,et al.  Training and analyzing deep recurrent neural networks , 2013, NIPS 2013.

[22]  Charles W. Chase Measuring Forecast Accuracy , 1995 .

[23]  Peter Stagge,et al.  Recurrent neural networks for time series classification , 2003, Neurocomputing.

[24]  Sahin Albayrak,et al.  Pattern recognition and classification for multivariate time series , 2011, SensorKDD '11.

[25]  Mollaiy Berneti Shahram,et al.  An Imperialist Competitive Algorithm Artificial Neural Network Method to Predict Oil Flow Rate of the Wells , 2011 .

[26]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[27]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[28]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[29]  Xindong Wu,et al.  10 Challenging Problems in Data Mining Research , 2006, Int. J. Inf. Technol. Decis. Mak..

[30]  Marc Parizeau,et al.  DEAP: evolutionary algorithms made easy , 2012, J. Mach. Learn. Res..

[31]  Michael Y. Hu,et al.  Forecasting with artificial neural networks: The state of the art , 1997 .

[32]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[33]  Rob J Hyndman,et al.  25 years of time series forecasting , 2006 .

[34]  Benjamin Schrauwen,et al.  Training and Analysing Deep Recurrent Neural Networks , 2013, NIPS.