An Extended Evaluation of Two-Phase Scheduling Methods for Animation Rendering

Recently HP Labs engaged in a joint project with DreamWorks Animation to develop a Utility Rendering Service that was used to render part of the computer-animated feature film Shrek 2. In a companion paper [2] we formalized the problem of scheduling animation rendering jobs and demonstrated that the general problem is computationally intractable, as are severely restricted special cases. We presented a novel and efficient two-phase scheduling method and evaluated it both theoretically and via simulation using large and detailed traces collected in DreamWorks Animation's production environment. In this paper we describe the overall experience of the joint project and greatly expand our empirical evaluations of job scheduling strategies for improving scheduling performance. Our new results include a workload characterization of DreamWorks Animation animation rendering jobs. We furthermore present parameter sensitivity analyses based on simulations using randomly generated synthetic workloads. Whereas our previous theoretical results imply that worst-case performance can be far from optimal for certain workloads, our current empirical results demonstrate that our scheduling method achieves performance quite close to optimal for both real and synthetic workloads. We furthermore offer advice for tuning a parameter associated with our method. Finally, we report a surprising performance anomaly involving a workload parameter that our previous theoretical analysis identified as crucial to performance. Our results also shed light on performance tradeoffs surrounding task parallelization.

[1]  Ronald L. Graham,et al.  Bounds for certain multiprocessing anomalies , 1966 .

[2]  Ronald L. Graham,et al.  Bounds on Multiprocessing Timing Anomalies , 1969, SIAM Journal of Applied Mathematics.

[3]  Richard P. Brent,et al.  The Parallel Evaluation of General Arithmetic Expressions , 1974, JACM.

[4]  K. Mani Chandy,et al.  A comparison of list schedules for parallel processing systems , 1974, Commun. ACM.

[5]  Anurag Kumar,et al.  Performance Analysis and Scheduling of Stochastic Fork-Join Jobs in a Multicomputer System , 1993, IEEE Trans. Parallel Distributed Syst..

[6]  Michael Pinedo,et al.  Scheduling: Theory, Algorithms, and Systems , 1994 .

[7]  David A. Lifka,et al.  The ANL/IBM SP Scheduling System , 1995, JSSPP.

[8]  Jirí Sgall,et al.  On-line Scheduling , 1996, Online Algorithms.

[9]  Uwe Schwiegelshohn,et al.  Theory and Practice in Parallel Job Scheduling , 1997, JSSPP.

[10]  Jiri Sgall,et al.  On-line scheduling --- a survey , 1997 .

[11]  Uwe Schwiegelshohn,et al.  On the Design and Evaluation of Job Scheduling Algorithms , 1999, JSSPP.

[12]  Heinrich Müller,et al.  Effiziente Methoden der geometrischen Modellierung und der wissenschaftlichen Visualisierung, Dagstuhl Seminar 1997 , 1999, Effiziente Methoden der geometrischen Modellierung und der wissenschaftlichen Visualisierung.

[13]  Y.-K. Kwok,et al.  Static scheduling algorithms for allocating directed task graphs to multiprocessors , 1999, CSUR.

[14]  Miron Livny,et al.  Adaptive Scheduling for Master-Worker Applications on the Computational Grid , 2000, GRID.

[15]  D. Atkin OR scheduling algorithms. , 2000, Anesthesiology.

[16]  Rajkumar Buyya,et al.  Grid Computing — GRID 2000 , 2002, Lecture Notes in Computer Science.

[17]  Zahir Tari,et al.  Dynamic task assignment in server farms: better performance by task grouping , 2002, Proceedings ISCC 2002 Seventh International Symposium on Computers and Communications.

[18]  Adam Wierman,et al.  Classifying scheduling policies with respect to unfairness in an M/GI/1 , 2003, SIGMETRICS '03.

[19]  Lev Markov Two stage optimization of job scheduling and assignment in heterogeneous compute farms , 2004, Proceedings. 10th IEEE International Workshop on Future Trends of Distributed Computing Systems, 2004. FTDCS 2004..

[20]  Dirk Beyer,et al.  Value-maximizing deadline scheduling and its application to animation rendering , 2005, SPAA '05.