Large-Scale Computing Systems Workload Prediction Using Parallel Improved LSTM Neural Network

In recent years, large-scale computing systems have been widely used as an important part of the computing infrastructure. Resource management based on systems workload prediction is an effective way to improve application efficiency. However, accuracy and real-time functionalities are always the key challenges that perplex the systems workload prediction model. In this paper, we first investigate the dependence on historical workload in large-scale computing systems and build a day and time two-dimensional time-series workload model. We then design a two-dimensional long short-term memory (LSTM) neural network cell structure. Based on this, we propose an improved LSTM prediction model providing its mathematical description and an error back propagation method. Furthermore, to achieve systems resource management real-time requirement, we provide a parallel improved LSTM algorithm that uses a hidden layer week-based dependence and weights parallelization algorithm. The comparative studies, based on the actual workload of the Shanghai Supercomputer Center, demonstrate that our proposed improved LSTM neural network prediction model can achieve higher accuracy and real-time performance in large-scale computing systems.

[1]  Billy M. Williams,et al.  Modeling and Forecasting Vehicular Traffic Flow as a Seasonal ARIMA Process: Theoretical Basis and Empirical Results , 2003, Journal of Transportation Engineering.

[2]  Bo Cheng,et al.  A cost-aware auto-scaling approach using the workload prediction in service clouds , 2014, Inf. Syst. Frontiers.

[3]  Peng Wang,et al.  High-Frequency Nonlinear Earthquake Simulations on Petascale Heterogeneous Supercomputers , 2016, SC16: International Conference for High Performance Computing, Networking, Storage and Analysis.

[4]  Xiaoyong Tang,et al.  Application-aware deadline constraint job scheduling mechanism on large-scale computational grid , 2018, PloS one.

[5]  Mi Zhang,et al.  A hybrid application of soft computing methods with wavelet SVM and neural network to electric power load forecasting , 2018 .

[6]  Jitendra Kumar,et al.  Workload prediction in cloud using artificial neural network and adaptive differential evolution , 2018, Future Gener. Comput. Syst..

[7]  Shaifu Gupta,et al.  Resource usage prediction of cloud workloads using deep bidirectional long short term memory networks , 2017, 2017 IEEE International Conference on Advanced Networks and Telecommunications Systems (ANTS).

[8]  Yasushi Inoguchi,et al.  Improving accuracy of host load predictions on computational grids by artificial neural networks , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[9]  Jie Zheng,et al.  Energy efficient job scheduling with workload prediction on cloud data center , 2018, Cluster Computing.

[10]  Raffaela Mirandola,et al.  An online learning model based on episode mining for workload prediction in cloud , 2018, Future Gener. Comput. Syst..

[11]  Nidhi Singh,et al.  Online Ensemble Learning Approach for Server Workload Prediction in Large Datacenters , 2012, 2012 11th International Conference on Machine Learning and Applications.

[12]  Igor N. Aizenberg,et al.  Towards Quantum Computing Algorithms for Datacenter Workload Predictions , 2018, 2018 IEEE 11th International Conference on Cloud Computing (CLOUD).

[13]  Wonyong Sung,et al.  Single stream parallelization of generalized LSTM-like RNNs on a GPU , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  Mascha C. van der Voort,et al.  Combining kohonen maps with arima time series models to forecast traffic flow , 1996 .

[15]  New high performance computing software for multiphysics simulations of fusion reactors , 2018, Fusion Engineering and Design.

[16]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[17]  Sheng Di,et al.  Google hostload prediction based on Bayesian model with optimized feature combination , 2014, J. Parallel Distributed Comput..

[18]  Matjaz B. Juric,et al.  AME-WPC: Advanced model for efficient workload prediction in the cloud , 2015, J. Netw. Comput. Appl..

[19]  Mostafa Ghobaei-Arani,et al.  A learning automata-based ensemble resource usage prediction algorithm for cloud computing environment , 2018, Future Gener. Comput. Syst..

[20]  Wenhao Huang,et al.  Deep Architecture for Traffic Flow Prediction: Deep Belief Networks With Multitask Learning , 2014, IEEE Transactions on Intelligent Transportation Systems.

[21]  Evangelos E. Milios,et al.  A Lightweight Algorithm for Message Type Extraction in System Application Logs , 2012, IEEE Transactions on Knowledge and Data Engineering.

[22]  Bo Li,et al.  Workload Prediction for Cloud Cluster Using a Recurrent Neural Network , 2016, 2016 International Conference on Identification, Information and Knowledge in the Internet of Things (IIKI).

[23]  Yu Chen,et al.  Quantitative workload analysis and prediction using Google cluster traces , 2016, 2016 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[24]  Han Cao,et al.  Prediction for Tourism Flow based on LSTM Neural Network , 2017, IIKI.

[25]  Rishabh Madan,et al.  Predicting Computer Network Traffic: A Time Series Forecasting Approach Using DWT, ARIMA and RNN , 2018, 2018 Eleventh International Conference on Contemporary Computing (IC3).

[26]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[27]  S. M. Aqil Burney,et al.  Forecasting Network Traffic Load Using Wavelet Filters and Seasonal Autoregressive Moving Average Model , 2010 .

[28]  Timothy R. Anderson,et al.  Technological Forecasting of Supercomputer Development: The March to Exascale Computing , 2015 .