An Architecture for Automatic Scaling of Replicated Services

Replicated services that allow to scale dynamically can adapt to requests load. Choosing the right number of replicas is fundamental to avoid performance worsening when input spikes occur and to save resources when the load is low. Current mechanisms for automatic scaling are mostly based on fixed thresholds on CPU and memory usage, which are not sufficiently accurate and often entail late countermeasures. We propose Make Your Service Elastic (MYSE), an architecture for automatic scaling of generic replicated services based on queuing models for accurate response time estimation. Requests and service times patterns are analyzed to learn and predict over time their distribution so as to allow for early scaling. A novel heuristic is proposed to avoid the flipping phenomenon. We carried out simulations that show promising results for what concerns the effectiveness of our approach.

[1]  Isis Truck,et al.  From Data Center Resource Allocation to Control Theory and Back , 2010, 2010 IEEE 3rd International Conference on Cloud Computing.

[2]  Michael I. Jordan,et al.  Statistical Machine Learning Makes Automatic Control Practical for Internet Datacenters , 2009, HotCloud.

[3]  Michael Y. Hu,et al.  Forecasting with artificial neural networks: The state of the art , 1997 .

[4]  Aniruddha S. Gokhale,et al.  Efficient Autoscaling in the Cloud Using Predictive Models for Workload Forecasting , 2011, 2011 IEEE 4th International Conference on Cloud Computing.

[5]  I. J. Myung,et al.  Tutorial on maximum likelihood estimation , 2003 .

[6]  Gang Yin,et al.  Online Self-Reconfiguration with Performance Guarantee for Energy-Efficient Large-Scale Cloud Computing Data Centers , 2010, 2010 IEEE International Conference on Services Computing.

[7]  David Clark,et al.  Safety and Security Analysis of Object-Oriented Models , 2002, SAFECOMP.

[8]  Rogério de Lemos,et al.  Architecting Dependable Systems VI , 2009, WADS.

[9]  Waheed Iqbal,et al.  Adaptive resource provisioning for read intensive multi-tier applications in the cloud , 2011, Future Gener. Comput. Syst..

[10]  Le Yi Wang,et al.  VCONF: a reinforcement learning approach to virtual machines auto-configuration , 2009, ICAC '09.

[11]  Sang-Min Park,et al.  Self-Tuning Virtual Machines for Predictable eScience , 2009, 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid.

[12]  Enda Barrett,et al.  Applying reinforcement learning towards automating resource allocation and application scalability in the cloud , 2013, Concurr. Comput. Pract. Exp..

[13]  Miguel Correia,et al.  Architecting Dependable Systems , 2003, Lecture Notes in Computer Science.

[14]  Moustafa Ghanem,et al.  Lightweight Resource Scaling for Cloud Applications , 2012, 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012).

[15]  Roberto Baldoni,et al.  Online Black-Box Failure Prediction for Mission Critical Distributed Systems , 2012, SAFECOMP.

[16]  Priya Narasimhan,et al.  Tiresias: Black-Box Failure Prediction in Distributed Systems , 2007, 2007 IEEE International Parallel and Distributed Processing Symposium.

[17]  Alexander Clemm,et al.  Integrated and autonomic cloud resource scaling , 2012, 2012 IEEE Network Operations and Management Symposium.

[18]  Xiaohui Gu,et al.  CloudScale: elastic resource scaling for multi-tenant cloud systems , 2011, SoCC.

[19]  Johan Tordsson,et al.  An adaptive hybrid elasticity controller for cloud infrastructures , 2012, 2012 IEEE Network Operations and Management Symposium.

[20]  Keikichi Hirose,et al.  Kolmogorov-Smirnov Test in Text-Dependent Automatic Speaker Identification , 2008, Eng. Lett..

[21]  Zhenhuan Gong,et al.  PRESS: PRedictive Elastic ReSource Scaling for cloud systems , 2010, 2010 International Conference on Network and Service Management.

[22]  Qi Zhang,et al.  A Regression-Based Analytic Model for Dynamic Resource Provisioning of Multi-Tier Applications , 2007, Fourth International Conference on Autonomic Computing (ICAC'07).

[23]  Jing Xu,et al.  On the Use of Fuzzy Modeling in Virtualized Data Center Management , 2007, Fourth International Conference on Autonomic Computing (ICAC'07).

[24]  Eddy Caron,et al.  Forecasting for Cloud computing on-demand resources based on pattern matching , 2010 .

[25]  Marin Litoiu,et al.  Optimal autoscaling in a IaaS cloud , 2012, ICAC '12.

[26]  Rajarshi Das,et al.  A Hybrid Reinforcement Learning Approach to Autonomic Resource Allocation , 2006, 2006 IEEE International Conference on Autonomic Computing.

[27]  Suman Nath,et al.  Energy-Aware Server Provisioning and Load Dispatching for Connection-Intensive Internet Services , 2008, NSDI.

[28]  Jinhui Huang,et al.  Resource prediction based on double exponential smoothing in cloud computing , 2012, 2012 2nd International Conference on Consumer Electronics, Communications and Networks (CECNet).

[29]  Abhishek Chandra,et al.  Resource Bundles: Using Aggregation for Statistical Large-Scale Resource Discovery and Management , 2010, IEEE Transactions on Parallel and Distributed Systems.

[30]  Isis Truck,et al.  Using Reinforcement Learning for Autonomic Resource Allocation in Clouds: towards a fully automated workflow , 2011 .

[31]  Kathryn Bean,et al.  Transforming reactive auto-scaling into proactive auto-scaling , 2013, CloudDP '13.

[32]  Eddy Caron,et al.  Forecasting for Grid and Cloud Computing On-Demand Resources Based on Pattern Matching , 2010, 2010 IEEE Second International Conference on Cloud Computing Technology and Science.

[33]  Prashant J. Shenoy,et al.  Agile dynamic provisioning of multi-tier Internet applications , 2008, TAAS.

[34]  Bradley R. Schmerl,et al.  Increasing System Dependability through Architecture-Based Self-Repair , 2002, WADS.

[35]  Rizos Sakellariou,et al.  Enacting SLAs in Clouds Using Rules , 2011, Euro-Par.

[36]  José Antonio Lozano,et al.  A Review of Auto-scaling Techniques for Elastic Applications in Cloud Environments , 2014, Journal of Grid Computing.

[37]  Dan Rubenstein,et al.  Provisioning servers in the application tier for e-commerce systems , 2004, Twelfth IEEE International Workshop on Quality of Service, 2004. IWQOS 2004..

[38]  Kevin Lee,et al.  Empirical prediction models for adaptive resource provisioning in the cloud , 2012, Future Gener. Comput. Syst..

[39]  Kang G. Shin,et al.  Automated control of multiple virtualized resources , 2009, EuroSys '09.

[40]  Emmanuel Jeannot,et al.  Euro-Par 2011 Parallel Processing , 2011, Lecture Notes in Computer Science.

[41]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .