Scheduling task parallel applications for rapid turnaround on desktop grids

Since the early 1990's, the largest distributed computing systems in the world have been desktop grids, which use the idle cycles of mainly desktop PC's to support large scale computation. Despite the enormous computing power offered by such systems, the range of supportable applications is largely limited to task parallel, compute-bound, and high-throughput applications. This limitation is mainly because of the heterogeneity and volatility of the underlying resources, which are shared with the desktop users. Our work focuses on broadening the applications supportable by desktop grids, and in particular, we focus on the development of scheduling heuristics to enable rapid turnaround for short-lived applications. To that end, the contributions of this dissertation are as follows. First, we measure and characterize four real enterprise desktop grid systems; such characterization is essential for accurate modelling and simulation. Second, using the characterization, we design scheduling heuristics that enable rapid application turnaround. These heuristics are based on three scheduling techniques, namely resource prioritization, resource exclusion, and task replication. We find that our best heuristic uses relatively static resource information for prioritization and exclusion, and reactive task replication to achieve performance within a factor of 1.7 of optimal. Third, we implement our best heuristic in a real desktop grid system to demonstrate its feasibility.