Predicting the Costs of Serverless Workflows

Function-as-a-Service (FaaS) platforms enable users to run arbitrary functions without being concerned about operational issues, while only paying for the consumed resources. Individual functions are often composed into workflows for complex tasks. However, the pay-per-use model and nontransparent reporting by cloud providers make it challenging to estimate the expected cost of a workflow, which prevents informed business decisions. Existing cost-estimation approaches assume a static response time for the serverless functions, without taking input parameters into account. In this paper, we propose a methodology for the cost prediction of serverless workflows consisting of input-parameter sensitive function models and a monte-carlo simulation of an abstract workflow model. Our approach enables workflow designers to predict, compare, and optimize the expected costs and performance of a planned workflow, which currently requires time-intensive experimentation. In our evaluation, we show that our approach can predict the response time and output parameters of a function based on its input parameters with an accuracy of 96.1%. In a case study with two audio-processing workflows, our approach predicts the costs of the two workflows with an accuracy of 96.2%.

[1]  Adam Eivy,et al.  Be Wary of the Economics of "Serverless" Cloud Computing , 2017, IEEE Cloud Computing.

[2]  Rajiv Ranjan,et al.  Resource and Performance Distribution Prediction for Large Scale Analytics Queries , 2016, ICPE.

[3]  Egor Bondarev,et al.  Modelling of input-parameter dependency for performance predictions of component-based embedded systems , 2005 .

[4]  Jóakim von Kistowski,et al.  Detecting Parametric Dependencies for Performance Models Using Feature Selection Techniques , 2019, 2019 IEEE 27th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS).

[5]  Lars Grunske,et al.  ArcheOpterix: An extendable tool for architecture optimization of AADL models , 2009, 2009 ICSE Workshop on Model-Based Methodologies for Pervasive and Embedded Software.

[6]  Cristina L. Abad,et al.  Reserved, on demand or serverless: Model-based simulations for cloud budget planning , 2017, 2017 IEEE Second Ecuador Technical Chapters Meeting (ETCM).

[7]  Rubén S. Montero,et al.  Dynamic placement of virtual machines for cost optimization in multi-cloud environments , 2011, 2011 International Conference on High Performance Computing & Simulation.

[8]  D. Vose.  Risk Analysis: A Quantitative Guide , 2000 .

[9]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[10]  Samuel Kounev,et al.  Modeling of Parametric Dependencies for Performance Prediction of Component-Based Software Systems at Run-Time , 2018, 2018 IEEE International Conference on Software Architecture (ICSA).

[11]  Ralf H. Reussner,et al.  Using Genetic Search for Reverse Engineering of Parametric Behavior Models for Performance Prediction , 2010, IEEE Transactions on Software Engineering.

[12]  S. Srihari.  Mixture Density Networks , 1994 .

[13]  Y. Yao,et al.  On Early Stopping in Gradient Descent Learning , 2007 .

[14]  Gul Agha,et al.  Costless: Optimizing Cost of Serverless Computing through Function Fusion and Placement , 2018, 2018 IEEE/ACM Symposium on Edge Computing (SEC).

[15]  Vasilios Andrikopoulos,et al.  Using a Microbenchmark to Compare Function as a Service Solutions , 2018, ESOCC.

[16]  Cristina L. Abad,et al.  The SPEC-RG Reference Architecture for FaaS: From Microservices and Containers to Serverless Platforms , 2019, IEEE Internet Computing.

[17]  Steffen Becker,et al.  Automatically improve software architecture models for performance, reliability, and cost using evolutionary algorithms , 2010, WOSP/SIPEW '10.

[18]  Geoffrey J. McLachlan,et al.  Mixture models : inference and applications to clustering , 1989 .

[19]  Wanda Niemyska,et al.  The Wasserstein Distance as a Dissimilarity Measure for Mass Spectra with Application to Spectral Deconvolution , 2018, WABI.

[20]  Md. Zakirul Alam Bhuiyan,et al.  A Survey on Deep Learning in Big Data , 2017, 22017 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC).

[21]  L. Ambrosio,et al.  Gradient Flows: In Metric Spaces and in the Space of Probability Measures , 2005 .

[22]  Joseph M. Hellerstein,et al.  Serverless Computing: One Step Forward, Two Steps Back , 2018, CIDR.

[23]  Peter H. N. de With,et al.  Modelling of input-parameter dependency for performance predictions of component-based embedded systems , 2005, 31st EUROMICRO Conference on Software Engineering and Advanced Applications.

[24]  Mehryar Mohri,et al.  L2 Regularization for Learning Kernels , 2009, UAI.

[25]  Alexandru Iosup,et al.  Serverless is More: From PaaS to Present Cloud Computing , 2018, IEEE Internet Computing.

[26]  Gojko Adzic,et al.  Serverless computing: economic and architectural impact , 2017, ESEC/SIGSOFT FSE.

[27]  Andrea C. Arpaci-Dusseau,et al.  Serverless Computation with OpenLambda , 2016, HotCloud.

[28]  Theo Lynn,et al.  A Preliminary Review of Enterprise Serverless Cloud Computing (Function-as-a-Service) Platforms , 2017, 2017 IEEE International Conference on Cloud Computing Technology and Science (CloudCom).

[29]  David A. Patterson,et al.  Cloud Programming Simplified: A Berkeley View on Serverless Computing , 2019, ArXiv.

[30]  Shaojie Tang,et al.  Towards Optimal Bidding Strategy for Amazon EC2 Cloud Spot Instance , 2012, 2012 IEEE Fifth International Conference on Cloud Computing.

[31]  George Kesidis,et al.  Spock: Exploiting Serverless Functions for SLO and Cost Aware Resource Procurement in Public Cloud , 2019, 2019 IEEE 12th International Conference on Cloud Computing (CLOUD).

[32]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[33]  L. Ambrosio,et al.  Chapter 1 – Gradient Flows of Probability Measures , 2007 .

[34]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[35]  Samuel Kounev,et al.  Blackbox Learning of Parametric Dependencies for Performance Models , 2018, MoDELS.

[36]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[37]  Alexandru Iosup,et al.  An Analysis of Workflow Formalisms for Workflows with Complex Non-Functional Requirements , 2018, ICPE Companion.

[38]  Cristina L. Abad,et al.  A SPEC RG Cloud Group's Vision on the Performance Challenges of FaaS Cloud Architectures , 2018, ICPE Companion.

[39]  Dirk P. Kroese,et al.  Kernel density estimation via diffusion , 2010, 1011.2602.

[40]  Perry Cheng,et al.  The serverless trilemma: function composition for serverless computing , 2017, Onward!.

[41]  Perry Cheng,et al.  Serverless Computing: Current Trends and Open Problems , 2017, Research Advances in Cloud Computing.

[42]  Bu-Sung Lee,et al.  Optimization of Resource Provisioning Cost in Cloud Computing , 2012, IEEE Transactions on Services Computing.

[43]  Norbert Siegmund,et al.  Transfer learning for performance modeling of configurable systems: An exploratory analysis , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[44]  José Luis Vázquez-Poletti,et al.  Serverless Computing: From Planet Mars to the Cloud , 2018, Computing in Science & Engineering.

[45]  Rolf Stadler,et al.  Predicting Distributions of Service Metrics using Neural Networks , 2018, 2018 14th International Conference on Network and Service Management (CNSM).

[46]  Yang Song,et al.  Optimal bidding in spot instance market , 2012, 2012 Proceedings IEEE INFOCOM.