Boosting HPC Applications in the Cloud Through JIT Traffic-Aware Path Provisioning

Data centers, clusters and grids have historically supported High-Performance Computing (HPC) applications. Due to the high capital and operational expenditures associated with such infrastructures, in recent past, we have witnessed consistent efforts to run HPC applications in the cloud. The potential advantages of this shift include higher scalability and lower costs. If on the one hand, app instantiation – through customized Virtual Machines (VMs) – is a well-solved issue, on the other, the network still represents a significant bottleneck. When switching HPC applications to be executed on the cloud, we lose control of where VMs will be positioned and of the paths that will be traversed for processes to communicate with one another. To alleviate this problem, and taking advantage of new advances in programmable networks, we propose a mechanism for dynamic, just-in-time path provisioning in cloud infrastructures. It continuously monitors the network conditions and, given the current communication patterns of the application, systematically (re)programs paths to avoid uncongested links and reduce end-to-end delays. The proposed mechanism achieves a speedup of up to 44.24% regarding application runtime when compared to the traditional shortest-path, static approach.