Photons: lambdas on a diet

Serverless computing allows users to create short, stateless functions and invoke hundreds of them concurrently to tackle massively parallel workloads. We observe that even though most of the footprint of a serverless function is fixed across its invocations --- language runtime, libraries, and other application state --- today's serverless platforms do not exploit this redundancy. Such an inefficiency has cascading negative impacts: longer startup times, lower throughput, higher latency, and higher cost. To mitigate these problems, we have built Photons, a framework leveraging workload parallelism to co-locate multiple instances of the same function within the same runtime. Concurrent invocations can then share the runtime and application state transparently, without compromising execution safety. Photons reduce function's memory consumption by 25% to 98% per invocation, with no performance degradation compared to today's serverless platforms. We also show that our approach can reduce the overall memory utilization by 30%, and the total number of cold starts by 52%.

[1]  Perry Cheng,et al.  Serverless Computing: Current Trends and Open Problems , 2017, Research Advances in Cloud Computing.

[2]  Yubin Xia,et al.  Catalyzer: Sub-millisecond Startup for Serverless Computing with Initialization-less Booting , 2020, ASPLOS.

[3]  Andrea C. Arpaci-Dusseau,et al.  SOCK: Rapid Task Provisioning with Serverless-Optimized Containers , 2018, USENIX Annual Technical Conference.

[4]  Anirudh Sivaraman,et al.  Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads , 2017, NSDI.

[5]  Paulo Ferreira,et al.  Dynamic vertical memory scalability for OpenJDK cloud applications , 2018, ISMM.

[6]  Ricardo Bianchini,et al.  Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider , 2020, USENIX Annual Technical Conference.

[7]  Florian Schmidt,et al.  My VM is Lighter (and Safer) than your Container , 2017, SOSP.

[8]  Dan Williams,et al.  Will Serverless End the Dominance of Linux in the Cloud? , 2017, HotOS.

[9]  Gustavo Alonso,et al.  Lambada: Interactive Data Analytics on Cold Data Using Serverless Cloud Infrastructure , 2020, SIGMOD Conference.

[10]  Eddie Kohler,et al.  Alto: lightweight VMs using virtualization-aware managed runtimes , 2018, ManLang '18.

[11]  Dirk Merkel,et al.  Docker: lightweight Linux containers for consistent development and deployment , 2014 .

[12]  George Kesidis,et al.  Spock: Exploiting Serverless Functions for SLO and Cost Aware Resource Procurement in Public Cloud , 2019, 2019 IEEE 12th International Conference on Cloud Computing (CLOUD).

[13]  Thierry Marianne Cloud Computing without Containers , 2018 .

[14]  Raul Castro Fernandez,et al.  Starling: A Scalable Query Engine on Cloud Functions , 2019, SIGMOD Conference.

[15]  Alfonso Pérez,et al.  Serverless computing for container-based architectures , 2018, Future Gener. Comput. Syst..

[16]  David G. Andersen,et al.  Putting the "Micro" Back in Microservice , 2018, USENIX Annual Technical Conference.

[17]  Muga Nishizawa,et al.  An Easy-to-Use Toolkit for Efficient Java Bytecode Translators , 2003, GPCE.

[18]  Joao Carreira,et al.  A Case for Serverless Machine Learning , 2018 .

[19]  Alexandru Agache,et al.  Firecracker: Lightweight Virtualization for Serverless Applications , 2020, NSDI.

[20]  Andrea C. Arpaci-Dusseau,et al.  Slacker: Fast Distribution with Lazy Docker Containers , 2016, FAST.

[21]  Randall L. Hyde Overview of memory management , 1988 .

[22]  Geoffrey M. Voelker,et al.  Sprocket: A Serverless Video Processing Framework , 2018, SoCC.

[23]  Christoforos E. Kozyrakis,et al.  Pocket: Elastic Ephemeral Storage for Serverless Analytics , 2018, OSDI.

[24]  Rüdiger Kapitza,et al.  Trust more, serverless , 2019, SYSTOR.

[25]  Vatche Ishakian,et al.  Serving Deep Learning Models in a Serverless Platform , 2017, 2018 IEEE International Conference on Cloud Engineering (IC2E).

[26]  Han Dong,et al.  SEUSS: skip redundant paths to make serverless fast , 2020, EuroSys.

[27]  Ion Stoica,et al.  Occupy the cloud: distributed computing for the 99% , 2017, SoCC.

[28]  Paarijaat Aditya,et al.  SAND: Towards High-Performance Serverless Computing , 2018, USENIX Annual Technical Conference.

[29]  Joseph M. Hellerstein,et al.  Serverless Computing: One Step Forward, Two Steps Back , 2018, CIDR.

[30]  Mengyuan Li,et al.  Peeking Behind the Curtains of Serverless Platforms , 2018, USENIX Annual Technical Conference.

[31]  Kshitij Doshi,et al.  Agile Cold Starts for Scalable Serverless , 2019, HotCloud.

[32]  G. Amdhal,et al.  Validity of the single processor approach to achieving large scale computing capabilities , 1967, AFIPS '67 (Spring).

[33]  Eyal de Lara,et al.  Poster Abstract: Hierarchical Serverless Computing for the Mobile Edge , 2016, 2016 IEEE/ACM Symposium on Edge Computing (SEC).

[34]  Yuan He,et al.  An Open-Source Benchmark Suite for Microservices and Their Hardware-Software Implications for Cloud & Edge Systems , 2019, ASPLOS.

[35]  Guihai Chen,et al.  KylinX: A Dynamic Library Operating System for Simplified and Efficient Cloud Virtualization , 2018, USENIX Annual Technical Conference.

[36]  Jon Crowcroft,et al.  Unikernels: library operating systems for the cloud , 2013, ASPLOS '13.

[37]  Randy H. Katz,et al.  Cirrus: a Serverless Framework for End-to-end ML Workflows , 2019, SoCC.