Vision of Scheduling in Parallel Systems

In this chapter, we examine vision of scheduling parallel applications which exist at various levels of computer systems. In other words this chapter is dedicated to the technical foundations of scheduling in parallel computing. We will study three elements of a parallel system: hardware, programming environment, i.e. programmer abstraction of the parallel system, and runtime environments. The purpose of this study is to find clues on scheduling problems that need to be solved, the approaches that may be viable, and limitations imposed on scheduling models. The following issues may be interesting: Is there any particular support for scheduling parallel applications? How much scheduling flexibility is left to a user of the application? Can the application be suspended, assigned to a selected processor, or migrated to a different processor? Are there any guarantees of executing parts of the application in parallel in real time? What data can be practically collected for the scheduling algorithms?

[1]  Rajesh Raman,et al.  Matchmaking: distributed resource management for high throughput computing , 1998, Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244).

[2]  Roger W. Hockney,et al.  The Communication Challenge for MPP: Intel Paragon and Meiko CS-2 , 1994, Parallel Computing.

[3]  Angel L. DeCegama,et al.  The technology of parallel processing: parallel processing architectures and VLSI hardware (vol. 1) , 1989 .

[4]  Abraham Silberschatz,et al.  Operating Systems Concepts , 2005 .

[5]  Bill Nitzberg,et al.  Noncontiguous Processor Allocation Algorithms for Mesh-Connected Multicomputers , 1997, IEEE Trans. Parallel Distributed Syst..

[6]  Esther M. Arkin,et al.  Processor allocation on Cplant: achieving general processor locality using one-dimensional allocation strategies , 2002, Proceedings. IEEE International Conference on Cluster Computing.

[7]  Bob Francis,et al.  Silicon Graphics Inc. , 1993 .

[8]  Jack J. Dongarra,et al.  Basic Linear Algebra Subprograms Technical (Blast) Forum Standard (1) , 2002, Int. J. High Perform. Comput. Appl..

[9]  Uwe Schwiegelshohn,et al.  Attributes for communication between Grid scheduling instances , 2004 .

[10]  Henri Casanova,et al.  Multiround algorithms for scheduling divisible loads , 2005, IEEE Transactions on Parallel and Distributed Systems.

[11]  Kai Hwang,et al.  Advanced computer architecture - parallelism, scalability, programmability , 1992 .

[12]  E. Lusk,et al.  Goals guiding design: PVM and MPI , 2002, Proceedings. IEEE International Conference on Cluster Computing.

[13]  D. B. Davis,et al.  Sun Microsystems Inc. , 1993 .

[14]  Mark J. Clement,et al.  Core Algorithms of the Maui Scheduler , 2001, JSSPP.

[15]  Jennifer M. Schopf,et al.  PBS Pro: Grid computing and scheduling attributes , 2004 .

[16]  B.R. Seyfarth,et al.  Glenda: an environment for easy parallel programming , 1994, Proceedings of IEEE Scalable High Performance Computing Conference.

[17]  Ming Q. Xu.  Effective metacomputing using LSF Multicluster , 2001, Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid.

[18]  Jack J. Dongarra,et al.  The PVM Concurrent Computing System: Evolution, Experiences, and Trends , 1994, Parallel Comput..

[19]  Nigel P. Topham,et al.  Performance of the decoupled ACRI-1 architecture: the perfect club , 1995, HPCN Europe.

[20]  Miltos D. Grammatikakis,et al.  Parallel System Interconnections and Communications , 2000 .

[21]  Tarek A. El-Ghazawi,et al.  Guest Editors' Introduction: High-Performance Reconfigurable Computing , 2007, Computer.

[22]  Yousef Saad,et al.  Data communication in parallel architectures , 1989, Parallel Comput..

[23]  Andrew S. Tanenbaum,et al.  Distributed operating systems , 2009, CSUR.

[24]  Jarek Nabrzyski,et al.  Grid Resource Management , 2004 .

[25]  Shahid H. Bokhari,et al.  Balancing contention and synchronization on the Intel Paragon , 1997, IEEE Concurrency.

[26]  Jarek Nabrzyski,et al.  Grid resource management: state of the art and future trends , 2004 .

[27]  P. Sadayappan,et al.  Selective buddy allocation for scheduling parallel jobs on clusters , 2002, Proceedings. IEEE International Conference on Cluster Computing.

[28]  Thomas G. Robertazzi,et al.  Distributed computation with communication delay (distributed intelligent sensor networks) , 1988 .

[29]  Jack Dongarra,et al.  Preface: Basic Linear Algebra Subprograms Technical (Blast) Forum Standard , 2002 .

[30]  Jacek Blazewicz,et al.  Distributed Processing of Divisible Jobs with Communication Startup Costs , 1997, Discret. Appl. Math..

[31]  Lionel M. Ni,et al.  A survey of wormhole routing techniques in direct networks , 1993, Computer.

[32]  Edward D. Lazowska,et al.  The Effect of Scheduling Discipline on Spin Overhead in Shared Memory Parallel Systems , 1991, IEEE Trans. Parallel Distributed Syst..

[33]  Daniel P. Siewiorek,et al.  Parallel processing: the Cm* experience , 1986 .

[34]  L Nelson Michael,et al.  A Comparison of Queueing, Cluster and Distributed Computing Systems , 1994 .

[35]  Zarza López,et al.  Chapley Gray John, Interprocess Communications in UNIX, Prentice Hall, Primera edición 1997 , 2002 .

[36]  Ramesh Subramonian,et al.  LogP: a practical model of parallel computation , 1996, CACM.

[37]  Ed Anderson,et al.  LAPACK Users' Guide , 1995 .

[38]  Domenico Talia,et al.  Performance Evaluation and Modeling of MPI Communications on the Meiko CS-2 , 1998, HPCN Europe.

[39]  David Notkin,et al.  Computer science in Japanese universities , 1993, Computer.

[40]  Larry Rudolph,et al.  Gang Scheduling Performance Benefits for Fine-Grain Synchronization , 1992, J. Parallel Distributed Comput..

[41]  Nicholas Carriero,et al.  Linda in context , 1989, CACM.

[42]  Ian Foster,et al.  Languages for Parallel Processing , 2000, Handbook on Parallel and Distributed Processing.

[43]  Richard N. Lagerstrom,et al.  PScheD: Political Scheduling on the CRAY T3E , 1997, JSSPP.

[44]  Nawaf Bitar,et al.  A Scalable Multi-Discipline, Multiple-Processor Scheduling Framework for IRIX , 1995, JSSPP.

[45]  David B. Jackson.  Grid scheduling with Maui/Silver , 2004 .

[46]  Luca Becchetti,et al.  Average case and smoothed competitive analysis of the multi-level feedback algorithm , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[47]  Miron Livny,et al.  Condor and preemptive resume scheduling , 2004 .

[48]  Edward G. Coffman,et al.  Feedback Queueing Models for Time-Shared Systems , 1968, J. ACM.

[49]  Edward G. Coffman,et al.  Synthesis of a Feedback Queueing Discipline for Computer Operation , 1974, JACM.

[50]  John K. Ousterhout.  Scheduling Techniques for Concurrebt Systems. , 1982, ICDCS 1982.

[51]  Chris Smith,et al.  Scheduling attributes and platform LSF , 2004 .

[52]  Jens Mache,et al.  Communication patterns and allocation strategies , 2004 .

[53]  Angel L. DeCegama,et al.  Parallel processing architectures and VLSI hardware , 1989 .

[54]  David A. Lifka,et al.  The ANL/IBM SP Scheduling System , 1995, JSSPP.

[55]  Jennifer M. Schopf,et al.  Ten Actions When Grid Scheduling , 2004 .

[56]  Roger W. Hockney,et al.  Performance parameters and benchmarking of supercomputers , 1991, Parallel Comput..

[57]  Michael G. Norman,et al.  Models of machines and computation for mapping in multicomputers , 1993, CSUR.

[58]  Henri Casanova,et al.  Network modeling issues for grid application scheduling , 2005, Int. J. Found. Comput. Sci..

[59]  Michael J. Flynn,et al.  Very high-speed computing systems , 1966 .

[60]  Ii Robert G. Babb.  Programming parallel processors , 1987 .

[61]  Pawel Wolniewicz,et al.  Experiments with Scheduling Divisible Tasks in Clusters of Workstations , 2000, Euro-Par.

[62]  Paul D. Gader,et al.  Image algebra techniques for parallel image processing , 1987 .