Job Characteristics of a Production Parallel Scientivic Workload on the NASA Ames iPSC/860

Statistics of a parallel workload on a 128-node iPSC/860 located at NASA Ames are presented. It is shown that while the number of sequential jobs dominates the number of parallel jobs, most of the resources (measured in node-seconds) were consumed by parallel jobs. Moreover, most of the sequential jobs were for system administration. The average runtime of jobs grew with the number of nodes used, so the total resource requirements of large parallel jobs were larger by more than the number of nodes they used. The job submission rate during peak day activity was somewhat lower than one every two minutes, and the average job size was small. At night, submission rate was low but job sizes and system utilization were high, mainly due to NQS. Submission rate and utilization over the weekend were lower than on weekdays. The overall utilization was 50%, after accounting for downtime. About 2/3 of the applications were executed repeatedly, some for a significant number of times.

[1]  Manoj Kumar,et al.  Measuring Parallelism in Computation-Intensive Scientific/Engineering Applications , 1988, IEEE Trans. Computers.

[2]  John L. Gustafson,et al.  Reevaluating Amdahl's law , 1988, CACM.

[3]  Shikharesh Majumdar,et al.  Characterisation of Programs for Scheduling in Multiprogrammed Parallel Systems , 1991, Perform. Evaluation.

[4]  Philip Heidelberger,et al.  Computer Performance Evaluation Methodology , 1984, IEEE Transactions on Computers.

[5]  Ashok K. Agrawala,et al.  An Approach to the Workload Characterization Problem , 1976, Computer.

[6]  Kenneth C. Sevcik.  Characterizations of parallelism in applications and their use in scheduling , 1989, SIGMETRICS '89.

[7]  Larry Rudolph,et al.  Wasted resources in gang scheduling , 1990, Proceedings of the 5th Jerusalem Conference on Information Technology, 1990. 'Next Decade in Information Technology'.

[8]  G. Amdhal,et al.  Validity of the single processor approach to achieving large scale computing capabilities , 1967, AFIPS '67 (Spring).

[9]  R. Chawla,et al.  The Stealth distributed scheduler , 1991, [1991] Proceedings. 11th International Conference on Distributed Computing Systems.

[10]  D. B. Davis,et al.  Intel Corp. , 1993 .

[11]  Mark S. Squillante,et al.  Performance analysis of job scheduling policies in parallel supercomputing environments , 1993, Supercomputing '93. Proceedings.

[12]  Robert E. Benner,et al.  Development of Parallel Methods for a $1024$-Processor Hypercube , 1988 .

[13]  Domenico Ferrari,et al.  Workload charaterization and Selection in Computer Performance Measurement , 1972, Computer.

[14]  Mary K. Vernon,et al.  Use of application characteristics and limited preemption for run-to-completion parallel processor scheduling policies , 1994, SIGMETRICS.

[15]  Lionel M. Ni,et al.  Scalable Problems and Memory-Bounded Speedup , 1993, J. Parallel Distributed Comput..

[16]  Giuseppe Serazzi,et al.  A Characterization of the Variation in Time of Workload Arrival Patterns , 1985, IEEE Transactions on Computers.

[17]  Patrick H. Worley,et al.  The Effect of Time Constraints on Scaled Speedup , 1990, SIAM J. Sci. Comput..

[18]  Ravishankar K. Iyer,et al.  Predictability of Process Resource Usage: A Measurement-Based Study on UNIX , 1989, IEEE Trans. Software Eng..

[19]  Shikharesh Majumdar,et al.  Scheduling in multiprogrammed parallel systems , 1988, SIGMETRICS 1988.

[20]  E. Barszcz.  Intercube communication for the iPSC/860 , 1992, Proceedings Scalable High Performance Computing Conference SHPCC-92..

[21]  David Kotz,et al.  Dynamic file-access characteristics of a production parallel scientific workload , 1994, Proceedings of Supercomputing '94.

[22]  Phillip Krueger,et al.  Processor allocation vs. job scheduling on hypercube computers , 1991, [1991] Proceedings. 11th International Conference on Distributed Computing Systems.

[23]  Anoop Gupta,et al.  Scaling parallel programs for multiprocessors: methodology and examples , 1993, Computer.

[24]  Giuseppe Serazzi,et al.  Workload characterization: a survey , 1993, Proc. IEEE.

[25]  Mary K. Vernon,et al.  The performance of multiprogrammed multiprocessor scheduling algorithms , 1990, SIGMETRICS '90.

[26]  Paul Messina.  The concurrent supercomputing consortium: Year 1 , 1993, IEEE Parallel & Distributed Technology: Systems & Applications.