Serverless architecture for workflow scheduling with unconstrained execution environment

Cloud computing, together with all its boons, has taken the world of computing by a storm, reaping benefits across various domains. Introduction of serverless computing, and more precisely Function-as-a-Service (FaaS), removed many orchestration and maintenance issues that system designers were facing. This inspired an emerging field of research on utilization and optimization of serverless computing. Existing body of work on this topic is, to the best of authors knowledge, focused on using serverless functions (e.g. AWS Lambda, Google Cloud Functions) exclusively. Such functions suffer constraints in the context of their execution environment, time or available space. In this paper we present a more general approach that improves upon existing architectures that revolve around cloud functions. By leveraging AWS Fargate technology, we propose a fully serverless and infinitely scalable architecture that is based on producer-consumer pattern and can be shaped to satisfy wide range of requirements. As worker nodes, Docker containers are used, which helps avoid aforementioned constrains. This concept is put to use in a system for acquisition of high frequency data.