Macro-level scheduling in the Cilk Network of Workstations environment

The term "macro-level scheduling" refers to finding and recruiting idle workstations and allocating them to various adaptively parallel applications. In this thesis, I have designed and implemented a macro-level scheduler for the Cilk Network of Workstations environment. Cilk-NOW provides the "micro-level scheduling" needed to allow programs to be executed adaptively in parallel on an unreliable network of workstations. This macro-level scheduler is designed to be hassle-free and easy to use and customize. It can tolerate network faults, and it can recover from workstation failures. Idleness can be defined in a highly flexible way, in order to minimize the chances of bothering a workstation's owner, but without losing valuable computation time. The security mechanism employed by the macroscheduler can be adapted to make running unauthorized code essentially as difficult as any given system's existing remote execution protocol makes it. This scheduler is also fair, in that it assigns jobs evenly to the set of available processors. Furthermore, the scheduling algorithm utilizes randomness and primarily local information at each processor to distribute the processors among the jobs. Extensive simulations suggest that this algorithm converges to within one processor per job of the fair allocation within O(log P) steps, where P is the number of processors. Thesis Supervisor: Charles E. Leiserson Title: Professor of Computer Science and Engineering