An extensible job scheduling system for massively parallel processor architectures

During the last five years scientists have discovered that modern UNIX workstations connected with ethernet and high performance networks can provide enough computational performance to compete with the supercomputers of the day. Today supercomputer systems, like International Business Machines SP, can provide more CPU and networking bandwidth than is obtainable from networks of workstations, NOWs. The IBM SP is actually made up of individual workstation class processors connected by a high bandwidth switch network so scheduler developers felt that the scheduling systems that were previously used on NOWs would still apply. It became obvious to the many sites who purchased MPP systems that this was certainly not the case. Realizing that there was an urgent need for a job scheduling system that works well in an MPP environment I started the development of the Extensible Argonne Scheduling sYstem (EASY). A unique development approach in which users were encouraged to make suggestions or report inconsistencies with the documented behavior of EASY was used. As EASY became more widely used and IBM SP systems started to become much larger, several scalability problems had to be addressed. The main scalability problem was due to the fact that EASY was doing resource management as well as job scheduling. After discussions with IBM on how to address these issues, we decided on an application programming interface to their LoadLeveler product. LoadLeveler did a poor job of scheduling large parallel jobs but did provide scaleable resource management. The EASY-LoadLeveler project was started to combine the best features of both scheduling systems. In order to test the scalability of EASY-LoadLeveler a much larger system than I had access to at Argonne was necessary. The Cornell Theory Center provided not only the largest IBM SP in existence at the time, but also a very close working relationship with IBM, making it an ideal place to do this research. Within a year we had the first version up and running at the Theory Center. This work has been extended to include a new deterministic heterogeneous scheduling algorithm that can be used on systems that have nodes with different resources.