Macaca: A Scalable and Energy-Efficient Platform for Coupling Cloud Computing with Distributed Embedded Computing

Microservers (embedded devices) with built-in sensors and network connectivity have become increasingly pervasive and their computational capabilities continue to be improved. Many works present that a heterogeneous cluster with the low-power microservers and high-performance nodes can provide competitive performance efficiency. However, they make simple modifications in existing distributed systems for adaptation, which have been proven not to fully exploit various the heterogeneous resources. In this paper, we argue that such heterogeneous cluster also calls for flexible and efficient computational resource scheduling. We then introduce Macaca, a platform for sharing and scheduling the distributed resources from embedded devices and Linux servers including computational resources, scale-out storages, and various data to accomplish collaborative processing tasks. In Macaca, we propose a two-layer scheduling mechanism to enhance utilization of these heterogeneous resources. Internally, the resource abstraction layer supports various sophisticated schedulers of existing distributed frameworks and decides how many resources to offer computing frameworks, while resource management layer is constructed for overall coordination of computational effectiveness and energy management for devices. Furthermore, Macaca adopts a novel strategy to support smart switch in three system models for energy-saving effectiveness. We evaluate Macaca by a variety of datasets and typical datacenter workloads, and the result shows that Macaca can achieve more efficient utilization of resources when sharing the heterogeneous cluster among diverse frameworks.