Memory conscious task partition and scheduling in grid environments

While resource management and task scheduling are identified challenges of grid computing, current grid scheduling systems mainly focus on CPU and network availability. Recent performance improvement of CPU and computer network has made memory usage a significant factor of overall performance. In this study, we consider memory availability as a performance factor and introduce memory conscious task partition and scheduling. Three task partition policies are discussed. They are CPU-based, memory-based, and CPU-memory combined partition. We first investigate the three task partition policies on dedicated resources and verify the effectiveness of the CPU-memory combined partition algorithm in finding an optimal solution. We then extend the task partition policies in nondedicated environments with the consideration of resource sharing. Analytical and experimental results show that the CPU-memory combined scheduling approach outperforms either the CPU-based or memory-based scheduling approach considerably for memory-intensive applications in grid environments.

[1]  Zhiling Lan,et al.  Dynamic Load Balancing of SAMR Applications on Distributed Systems , 2001, ACM/IEEE SC 2001 Conference (SC'01).

[2]  Ian Foster,et al.  A quality of service architecture that combines resource reservation and application adaptation , 2000, 2000 Eighth International Workshop on Quality of Service. IWQoS 2000 (Cat. No.00EX400).

[3]  Baruch Awerbuch,et al.  An Opportunity Cost Approach for Job Assignment in a Scalable Computing Cluster , 2000, IEEE Trans. Parallel Distributed Syst..

[4]  Francine Berman,et al.  Adaptive Computing on the Grid Using AppLeS , 2003, IEEE Trans. Parallel Distributed Syst..

[5]  Richard Wolski,et al.  The network weather service: a distributed resource performance forecasting service for metacomputing , 1999, Future Gener. Comput. Syst..

[6]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[7]  Miron Livny,et al.  Condor-a hunter of idle workstations , 1988, [1988] Proceedings. The 8th International Conference on Distributed.

[8]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[9]  Miron Livny,et al.  The Available Capacity of a Privately Owned Workstation Environmont , 1991, Perform. Evaluation.

[10]  Jeffrey K. Hollingsworth,et al.  Mechanisms and policies for supporting fine-grained cycle stealing , 1999, ICS '99.

[11]  David Abramson,et al.  Nimrod: a tool for performing parametrised simulations using distributed workstations , 1995, Proceedings of the Fourth IEEE International Symposium on High Performance Distributed Computing.

[12]  Henri Casanova,et al.  Adaptive Scheduling for Task Farming with Grid Middleware , 1999, Int. J. High Perform. Comput. Appl..

[13]  Henri Casanova,et al.  Netsolve: a Network-Enabled Server for Solving Computational Science Problems , 1997, Int. J. High Perform. Comput. Appl..

[14]  Ming Wu,et al.  A general self-adaptive task scheduling system for non-dedicated heterogeneous computing , 2003, 2003 Proceedings IEEE International Conference on Cluster Computing.

[15]  Li Xiao,et al.  Dynamic Cluster Resource Allocations for Jobs with Known and Unknown Memory Demands , 2002, IEEE Trans. Parallel Distributed Syst..

[16]  Baruch Awerbuch,et al.  An Opportunity Cost Approach for Job Assignment and Reassignment in a Scalable Computing Cluster , 2002 .

[17]  Xian-He Sun,et al.  Performance Modeling and Prediction of Nondedicated Network Computing , 2002, IEEE Trans. Computers.

[18]  Joel H. Saltz,et al.  The utility of exploiting idle workstations for parallel computation , 1997, SIGMETRICS '97.

[19]  David Abramson,et al.  Nimrod/G: an architecture for a resource management and scheduling system in a global computational grid , 2000, Proceedings Fourth International Conference/Exhibition on High Performance Computing in the Asia-Pacific Region.