Optimal job packing, a backfill scheduling optimization for a cluster of workstations

In this paper, we have proposed two backfill scheduling optimizations, i.e., Shortest Width First Backfill (SWFBF) and Shortest Area First Backfill (SAFBF). A near optimal simple, but effective job packing algorithm called the Select-Replace algorithm has also been presented to minimize external fragmentation. Proof of the concept has been given with the help of a simulation study. Five workloads which were derived from a clean version of the parallel workload archive (CTC, LANL, and SDSC. NASA) have been used to evaluate and compare proposed heuristics with previous techniques. With the simple but effective optimizations, significant (56.1%) performance improvement has been achieved as compared to EASY scheduler.

[1]  Rajkumar Buyya,et al.  GridSim: a toolkit for the modeling and simulation of distributed resource management and scheduling for Grid computing , 2002, Concurr. Comput. Pract. Exp..

[2]  Anand Sivasubramaniam,et al.  Improving parallel job scheduling by combining gang scheduling and backfilling techniques , 2000, Proceedings 14th International Parallel and Distributed Processing Symposium. IPDPS 2000.

[3]  Uwe Schwiegelshohn,et al.  Job Scheduling Strategies for Parallel Processing, 13th International Workshop, JSSPP 2007, Seattle, WA, USA, June 17, 2007. Revised Papers , 2003, JSSPP.

[4]  Dror G. Feitelson,et al.  Utilization and Predictability in Scheduling the IBM SP2 with Backfilling , 1998, Proceedings of the First Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing.

[5]  Jack Belzer,et al.  Encyclopedia of Computer Science and Technology , 2002 .

[6]  Mark Baker,et al.  Cluster Computing and Applications , 2000 .

[7]  Hugo Terashima-Marín,et al.  Comparing two models to generate hyper-heuristics for the 2d-regular bin-packing problem , 2007, GECCO '07.

[8]  Zhou Lei,et al.  The portable batch scheduler and the maui scheduler on linux clusters , 2000 .

[9]  Dan Tsafrir,et al.  Backfilling Using System-Generated Predictions Rather than User Runtime Estimates , 2007, IEEE Transactions on Parallel and Distributed Systems.

[10]  Uwe Schwiegelshohn,et al.  New Challenges of Parallel Job Scheduling , 2007, JSSPP.

[11]  Ricardo A. Baeza-Yates,et al.  Challenges on Distributed Web Retrieval , 2007, 2007 IEEE 23rd International Conference on Data Engineering.

[12]  Evgenia Smirni,et al.  Multiple-Queue Backfilling Scheduling with Priorities and Reservations for Parallel Systems , 2002, JSSPP.

[13]  Jun Ni,et al.  Application of the Linux cluster for exhaustive window haplotype analysis using the FBAT and Unphased programs , 2008, BMC Bioinformatics.

[14]  Dror G. Feitelson,et al.  Supporting priorities and improving utilization of the IBM SP scheduler using slack-based backfilling , 1999, Proceedings 13th International Parallel Processing Symposium and 10th Symposium on Parallel and Distributed Processing. IPPS/SPDP 1999.

[15]  Dror G. Feitelson,et al.  Backfilling with lookahead to optimize the packing of parallel jobs , 2005, J. Parallel Distributed Comput..

[16]  Dalibor Klusácek,et al.  Alea - Grid Scheduling Simulation Environment , 2007, PPAM.

[17]  Dan Tsafrir,et al.  Workload sanitation for performance evaluation , 2006, 2006 IEEE International Symposium on Performance Analysis of Systems and Software.

[18]  Dan Tsafrir,et al.  The Dynamics of Backfilling: Solving the Mystery of Why Increased Inaccuracy May Help , 2006, 2006 IEEE International Symposium on Workload Characterization.

[19]  P. Sadayappan,et al.  Selective Reservation Strategies for Backfill Job Scheduling , 2002, JSSPP.

[20]  Evgenia Smirni,et al.  Multiple-queue backfilling scheduling with priorities and reservations for parallel systems , 2002, PERV.

[21]  John E. West,et al.  Scheduling Jobs on Parallel Systems Using a Relaxed Backfill Strategy , 2002, JSSPP.

[22]  Andrea C. Arpaci-Dusseau,et al.  The Impact of More Accurate Requested Runtimes on Production Job Scheduling Performance , 2002, JSSPP.

[23]  James Patton Jones,et al.  Scheduling for Parallel Supercomputing: A Historical Perspective of Achievable Utilization , 1999, JSSPP.

[24]  Evgenia Smirni,et al.  Self-adapting backfilling scheduling for parallel systems , 2002, Proceedings International Conference on Parallel Processing.