Cost function based event triggered Model Predictive Controllers application to Big Data Cloud services

High rate cluster reconfigurations is a costly issue in Big Data Cloud services. Current control solutions manage to scale the cluster according to the workload, however they do not try to minimize the number of system reconfigurations. Event-based control is known to reduce the number of control updates typically by waiting for the system states to degrade below a given threshold before reacting. However, computer science systems often have exogenous inputs (such as clients connections) with delayed impacts that can enable to anticipate states degradation. In this paper, a novel event-triggered approach is presented. This triggering mechanism relies on a Model Predictive Controller and is defined upon the value of the optimal cost function instead of the state or output error. This controller reduces the number of control changes, in the normal operation mode, through constraints in the MPC formulation but also insures a very reactive behavior to changes of exogenous inputs. This novel control approach is evaluated using a model validated on a real Big Data system. The controller efficiently scales the cluster according to specifications, while reducing its reconfigurations.

[1]  Nicolas Marchand,et al.  A General Formula for Event-Based Stabilization of Nonlinear Systems , 2013, IEEE Transactions on Automatic Control.

[2]  Enrico Bini,et al.  On Lyapunov sampling for event-driven controllers , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[3]  Zhenhuan Gong,et al.  PRESS: PRedictive Elastic ReSource Scaling for cloud systems , 2010, 2010 International Conference on Network and Service Management.

[4]  Mihaly Berekmeri,et al.  Feedback Autonomic Provisioning for Guaranteeing Performance in MapReduce Systems , 2018, IEEE Transactions on Cloud Computing.

[5]  Weisong Shi,et al.  Workload characterization on a production Hadoop cluster: A case study on Taobao , 2012, 2012 IEEE International Symposium on Workload Characterization (IISWC).

[6]  Aman Kansal,et al.  Q-clouds: managing performance interference effects for QoS-aware clouds , 2010, EuroSys '10.

[7]  Jan Lunze,et al.  A state-feedback approach to event-based control , 2010, Autom..

[8]  José A. B. Fortes,et al.  On the Use of Machine Learning to Predict the Time and Resources Consumed by Applications , 2010, 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing.

[9]  Karthick Rajamani,et al.  Energy Management for Commercial Servers , 2003, Computer.

[10]  Xiaohui Gu,et al.  AGILE: Elastic Distributed Resource Scaling for Infrastructure-as-a-Service , 2013, ICAC.

[11]  H. Michalska,et al.  Receding horizon control of nonlinear systems , 1988, Proceedings of the 28th IEEE Conference on Decision and Control,.

[12]  Johan Tordsson,et al.  An adaptive hybrid elasticity controller for cloud infrastructures , 2012, 2012 IEEE Network Operations and Management Symposium.

[13]  Vasudeva Varma,et al.  Dynamic energy efficient data placement and cluster reconfiguration algorithm for MapReduce framework , 2012, Future Gener. Comput. Syst..

[14]  M. Alamir,et al.  On the stability of receding horizon control of nonlinear discrete-time systems , 1994 .

[15]  Karl Johan Åström,et al.  Event Based Control , 2008 .

[16]  Franck Cappello,et al.  Grid'5000: a large scale and highly reconfigurable grid experimental testbed , 2005, The 6th IEEE/ACM International Workshop on Grid Computing, 2005..

[17]  Karl-Erik Årzén,et al.  A simple event-based PID controller , 1999 .

[18]  Yushi Shen,et al.  Enabling the New Era of Cloud Computing: Data Security, Transfer, and Management , 2013 .

[19]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[20]  Dimos V. Dimarogonas,et al.  Event-triggered control for discrete-time systems , 2010, Proceedings of the 2010 American Control Conference.

[21]  Mor Harchol-Balter,et al.  AutoScale: Dynamic, Robust Capacity Management for Multi-Tier Data Centers , 2012, TOCS.

[22]  Johan Tordsson,et al.  Efficient provisioning of bursty scientific workloads on the cloud using adaptive elasticity control , 2012, ScienceCloud '12.

[23]  José Antonio Lozano,et al.  A Review of Auto-scaling Techniques for Elastic Applications in Cloud Environments , 2014, Journal of Grid Computing.

[24]  Nicolas Marchand,et al.  Further results on event-based PID controller , 2009, 2009 European Control Conference (ECC).

[25]  Mihaly Berekmeri,et al.  Modeling and control of cloud services : application to MapReduce performance and dependability , 2015 .

[26]  Mihaly Berekmeri,et al.  A Control Approach for Performance of Big Data Systems , 2014 .

[27]  A. Zahariev.  Google App Engine , 2009 .

[28]  T. A. De Ruiter,et al.  A workload model for MapReduce , 2012 .