Auto-Scaling System in Apache Spark Cluster Using Model-Based Deep Reinforcement Learning

[1]  Chung-Horng Lung,et al.  Towards an Autonomic Auto-scaling Prediction System for Cloud Resource Provisioning , 2015, 2015 IEEE/ACM 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems.

[2]  Claus Pahl,et al.  A Comparison of Reinforcement Learning Techniques for Fuzzy Cloud Auto-Scaling , 2017, 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID).

[3]  Pankaj Singh,et al.  Big Data: Technologies, Trends and Applications , 2015 .

[4]  Scott Shenker,et al.  Shark: fast data analysis using coarse-grained distributed memory , 2012, SIGMOD Conference.

[5]  Michael J. Franklin,et al.  Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing , 2012, NSDI.

[6]  Reynold Xin,et al.  Scaling Spark in the Real World: Performance and Usability , 2015, Proc. VLDB Endow..

[7]  Jürgen Schmidhuber,et al.  Multi-dimensional Recurrent Neural Networks , 2007, ICANN.

[8]  Ahmad Sharieh,et al.  Resource Utilization in Cloud Computing as an Optimization Problem , 2016 .

[9]  Santo Fortunato,et al.  Attention Decay in Science , 2015, J. Informetrics.

[10]  Scott Shenker,et al.  Spark: Cluster Computing with Working Sets , 2010, HotCloud.