A Two-Layer Water Demand Prediction System in Urban Areas Based on Micro-Services and LSTM Neural Networks

In recent years, scarce water resources became one of the main problems that endanger human species existence and the advancement of any nation. In this research, smart water meters were implemented, distributed, and installed in a regional area in Cairo while data were collected at uniform intervals then sent to the cloud instantly. The solution paradigm uses an Internet of Things (IoT) based on micro-services and containers. The design incorporates real-time streaming and infrastructure performance optimization to store data. A second layer to analyze the acquired data was used to model water consumption using Long Short-Term Memory (LSTM). The designed LSTM is validated and tested to be utilized in the forecast of future water demand. Moreover, two alternative machine learning methods, namely Support Vector Regression and Random Forest commonly utilized in time series forecasting applications, were used for a comparative analysis of which LSTM has proven to be superior. The proper integration of the system elements is the key to the proposed system success. Based on the success of the designed system, it can be applicable on a national scale. That can enable the optimal management of consumers’ demand and improve water infrastructure utilization. The proposed paradigm presents a testbed for various scenarios that can be used in water resources management.

[1]  V. Krotov,et al.  Addressing barriers to big data , 2017 .

[2]  David Bernstein,et al.  Containers and Cloud: From LXC to Docker to Kubernetes , 2014, IEEE Cloud Computing.

[3]  Greta Goracci,et al.  On the Ergodicity of First‐Order Threshold Autoregressive Moving‐Average Processes , 2018, Journal of Time Series Analysis.

[4]  Ramón Alcarria,et al.  An IoT based reference architecture for smart water management processes , 2015, J. Wirel. Mob. Networks Ubiquitous Comput. Dependable Appl..

[5]  Lawrence C McCandless,et al.  Evaluation of random forest regression and multiple linear regression for predicting indoor fine particulate matter concentrations in a highly polluted city. , 2019, Environmental pollution.

[6]  Ilyoung Chong,et al.  Microservices Model to Enhance the Availability of Data for Buildings Energy Efficiency Management Services , 2019 .

[7]  Talel Bessaoudi,et al.  Robust state and fault estimation for non-linear stochastic systems with unknown disturbances: a multi-step delayed solution , 2019 .

[8]  Joaquín Izquierdo,et al.  Hybrid regression model for near real-time urban water demand forecasting , 2017, J. Comput. Appl. Math..

[9]  Antonello Monti,et al.  A cloud-based smart metering infrastructure for distribution grid services and automation , 2017, Sustainable Energy, Grids and Networks.

[10]  M. Iqbal,et al.  Sensor Networks for Monitoring and Control of Water Distribution Systems , 2013 .

[11]  Ryno Laubscher,et al.  Time-series forecasting of coal-fired power plant reheater metal temperatures using encoder-decoder recurrent neural networks , 2019 .

[12]  Saeed Sharifian,et al.  A water cycle optimized wavelet neural network algorithm for demand prediction in cloud computing , 2019, Cluster Computing.

[13]  Li Li,et al.  Mid-to-long term wind and photovoltaic power generation prediction based on copula function and long short term memory network , 2019, Applied Energy.

[14]  Suresh Sankaranarayanan,et al.  IoT-based water demand forecasting and distribution design for smart city , 2019, Journal of Water and Climate Change.

[15]  Fredrik Wallin,et al.  Is real-time electricity pricing suitable for residential users without demand-side management? , 2016 .

[16]  J. Thøgersen,et al.  Feedback on Household Electricity Consumption: Learning and Social Influence Processes , 2011 .

[17]  Elpiniki I. Papageorgiou,et al.  Hybrid model for water demand prediction based on fuzzy cognitive maps and artificial neural networks , 2016, 2016 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[18]  C. Holt Author's retrospective on ‘Forecasting seasonals and trends by exponentially weighted moving averages’ , 2004 .

[19]  David S. Linthicum Moving to Autonomous and Self-Migrating Containers for Cloud Applications , 2016, IEEE Cloud Computing.

[20]  Somnath Murlidhar Lambe,et al.  Microcontroller based Automatic Water level Control System , 2016 .

[21]  Volker Coors,et al.  COMPARATIVE ANALYSIS OF LSTM, RF AND SVM ARCHITECTURES FOR PREDICTING WIND NATURE FOR SMART CITY PLANNING , 2019, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[22]  Simo Särkkä,et al.  Iterative statistical linear regression for Gaussian smoothing in continuous-time non-linear stochastic dynamic systems , 2018, Signal Process..

[23]  Xuesong Yan,et al.  Stock price prediction based on deep neural networks , 2019, Neural Computing and Applications.

[24]  George K. Karagiannidis,et al.  Big Data Analytics for Dynamic Energy Management in Smart Grids , 2015, Big Data Res..

[25]  Esteban Jove,et al.  Geothermal heat exchanger energy prediction based on time series and monitoring sensors optimization , 2019, Energy.

[26]  K. Sathish Kumar,et al.  A survey on residential Demand Side Management architecture, approaches, optimization models and methods , 2016 .

[27]  Hongjie Jia,et al.  Vector field-based support vector regression for building energy consumption prediction , 2019, Applied Energy.

[28]  Stenio F. L. Fernandes,et al.  SWAMP: an IoT-based Smart Water Management Platform for Precision Irrigation in Agriculture , 2018, 2018 Global Internet of Things Summit (GIoTS).

[29]  Xiaoxia Qi,et al.  A comparison of day-ahead photovoltaic power forecasting models based on deep learning neural network , 2019, Applied Energy.

[30]  N. B. Anuar,et al.  The rise of "big data" on cloud computing: Review and open research issues , 2015, Inf. Syst..

[31]  Jun Ma,et al.  Air quality prediction at new stations using spatially transferred bi-directional long short-term memory network. , 2020, The Science of the total environment.

[32]  Jun Feng,et al.  Artificial Intelligence Approaches for Urban Water Demand Forecasting: A Review , 2019, MLICOM.

[33]  M. Bildirici Economic Growth and Electricity Consumption: MS‐VAR and MS‐Granger Causality Analysis , 2013 .

[34]  Marco Jahn,et al.  Designing a Smart City Internet of Things Platform with Microservice Architecture , 2015, 2015 3rd International Conference on Future Internet of Things and Cloud.

[35]  Juha-Pekka Soininen,et al.  Smart Water Management Platform: IoT-Based Precision Irrigation for Agriculture † , 2019, Sensors.

[36]  Alejandro Jerez,et al.  Forecasting emergency admissions due to respiratory diseases in high variability scenarios using time series: A case study in Chile. , 2019, The Science of the total environment.

[37]  Ehsan Ardjmand,et al.  Water demand forecasting: review of soft computing methods , 2017, Environmental Monitoring and Assessment.

[38]  Yunlong Yu,et al.  Effective Neural Network Training With a New Weighting Mechanism-Based Optimization Algorithm , 2019, IEEE Access.

[39]  J. Adamowski,et al.  Coupling the maximum overlap discrete wavelet transform and long short-term memory networks for irrigation flow forecasting , 2019, Agricultural Water Management.

[40]  Ramchandra Desai Real Time Analysis using Hadoop , 2016 .

[41]  Bahram Gharabaghi,et al.  New insights into soil temperature time series modeling: linear or nonlinear? , 2019, Theoretical and Applied Climatology.

[42]  Vijaya Kamble,et al.  Comparison of Predictive Algorithms: Backpropagation, SVM, LSTM and Kalman Filter for Stock Market , 2019, 2019 Amity International Conference on Artificial Intelligence (AICAI).

[43]  Rafal Kasperowicz Electricity Consumption and Economic Growth : Evidence from Poland , 2014 .

[44]  Abdelkader Bousselham,et al.  The Internet of Energy: Smart Sensor Networks and Big Data Management for Smart Grid , 2015, FNC/MobiSPC.

[45]  Boqiang Lin,et al.  Electricity tariff reform and rebound effect of residential electricity consumption in China , 2013 .

[46]  Sheikh Mohammad Idrees,et al.  A Prediction Approach for Stock Market Volatility Based on Time Series Data , 2019, IEEE Access.

[47]  Francesco Archetti,et al.  Tuning hyperparameters of a SVM-based water demand forecasting system through parallel global optimization , 2019, Comput. Oper. Res..

[48]  Rajkumar Buyya,et al.  Distributed data stream processing and edge computing: A survey on resource elasticity and future directions , 2017, J. Netw. Comput. Appl..

[49]  Kanika Modi,et al.  Big Data Streaming with Spark , 2019 .

[50]  Sunil Kumar,et al.  Exploiting interoperable microservices in web objects enabled Internet of Things , 2017, 2017 Ninth International Conference on Ubiquitous and Future Networks (ICUFN).

[51]  Jakub Jurasz,et al.  Forecasting surface water level fluctuations of lake Serwy (Northeastern Poland) by artificial neural networks and multiple linear regression , 2017 .

[52]  A note on limiting distribution of the sample auto-covariance function for the first-order autoregressive (AR(1)) model , 2018, Communications in Statistics - Theory and Methods.

[53]  John P. McCrae,et al.  A Comparative Study of SVM and LSTM Deep Learning Algorithms for Stock Market Prediction , 2019, AICS.

[54]  Sifeng Liu,et al.  Comparison of China's primary energy consumption forecasting by using ARIMA (the autoregressive integrated moving average) model and GM(1,1) model , 2016 .

[55]  J. P. Shri Tharanyaa,et al.  THEFT IDENTIFICATION AND AUTOMATED WATER SUPPLY SYSTEM USING EMBEDDED TECHNOLOGY , 2013 .

[56]  Mininath R. Bendre,et al.  Analytics, challenges and applications in big data environment: a survey , 2016 .

[57]  Raúl Aquino-Santos,et al.  RiverCore: IoT Device for River Water Level Monitoring over Cellular Communications , 2019, Sensors.

[58]  Sherif M. Badr,et al.  Healthcare Cloud Integration using Distributed Cloud Storage and Hybrid Image Compression , 2013 .

[59]  Alex Sherstinsky,et al.  Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network , 2018, Physica D: Nonlinear Phenomena.

[60]  Gang Mu,et al.  Big Data Analytics in China's Electric Power Industry: Modern Information, Communication Technologies, and Millions of Smart Meters , 2018, IEEE Power and Energy Magazine.

[61]  Gholamreza Naser,et al.  Support Vector Machines in Urban Water Demand Forecasting Using Phase Space Reconstruction , 2017 .

[62]  Slawek Smyl,et al.  A hybrid method of exponential smoothing and recurrent neural networks for time series forecasting , 2020, International Journal of Forecasting.

[63]  N. Suganya Automated Water Distribution and Performance Monitoring System , 2014 .

[64]  Heeyoung Kim,et al.  A new metric of absolute percentage error for intermittent demand forecasts , 2016 .