Metascheduling and Heuristic Co-Allocation Strategies in Distributed Computing

In this paper, we address problems of efficient computing in distributed systems with non-dedicated resources including utility grid. There are global job flows from external users along with resource owner's local tasks upon the resource non-dedication condition. Competition for resource reservation between independent users, local and global job flows substantially complicates scheduling and the requirement to provide the necessary quality of service. A metascheduling concept, justified in this work, assumes a complex combination of job flow dispatching and application-level scheduling methods for parallel jobs, as well as resource sharing and consumption policies established in virtual organizations and based on economic principles. We introduce heuristic slot selection and co-allocation strategies for parallel jobs. They are formalized by given criteria and implemented by algorithms of linear complexity on an available slots number.

[1]  Francine Berman,et al.  High-performance schedulers , 1998 .

[2]  Victor V. Toporkov,et al.  Resource Selection Algorithms for Economic Scheduling in Distributed Systems , 2011, ICCS.

[3]  Andrew S. Grimshaw,et al.  Grid resource management in legion , 2004 .

[4]  Johan Tordsson,et al.  A standards‐based Grid resource brokering service supporting advance reservations, coallocation, and cross‐Grid interoperability , 2009, Concurr. Comput. Pract. Exp..

[5]  Jarek Nabrzyski,et al.  Multicriteria aspects of Grid resource management , 2004 .

[6]  Kavitha Ranganathan,et al.  Decoupling computation and data scheduling in distributed data-intensive applications , 2002, Proceedings 11th IEEE International Symposium on High Performance Distributed Computing.

[7]  Gerald Tesauro,et al.  Strategic sequential bidding in auctions using dynamic programming , 2002, AAMAS '02.

[8]  Valentin Cristea,et al.  Resource CoAllocation for Scheduling Tasks with Dependencies, in Grid , 2011, ArXiv.

[9]  Ian T. Foster,et al.  The anatomy of the grid: enabling scalable virtual organizations , 2001, Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid.

[10]  Henri Casanova,et al.  Scheduling mixed-parallel applications with advance reservations , 2008, HPDC '08.

[11]  Fernando Guirado,et al.  MIP Model Scheduling for Multi-Clusters , 2012, Euro-Par Workshops.

[12]  Rajkumar Buyya,et al.  Scheduling Parallel Applications on Utility Grids: Time and Cost Trade-off Management , 2009, ACSC.

[13]  Muthucumaru Maheswaran,et al.  A Synchronous Co-Allocation Mechanism for Grid Computing Systems , 2004, Cluster Computing.

[14]  David Abramson,et al.  Economic models for resource management and scheduling in Grid computing , 2002, Concurr. Comput. Pract. Exp..

[15]  Ian T. Foster,et al.  Condor-G: A Computation Management Agent for Multi-Institutional Grids , 2004, Cluster Computing.

[16]  Victor V. Toporkov,et al.  Application-Level and Job-Flow Scheduling: An Approach for Achieving Quality of Service in Distributed Computing , 2009, PaCT.

[17]  Douglas Thain,et al.  Distributed computing in practice: the Condor experience , 2005, Concurr. Pract. Exp..

[18]  Rajkumar Buyya,et al.  Environment-conscious scheduling of HPC applications on distributed Cloud-oriented data centers , 2011, J. Parallel Distributed Comput..

[19]  George N. Rouskas,et al.  Resource co-allocation for large-scale distributed environments , 2009, HPDC '09.

[20]  Henri Casanova,et al.  Multiround algorithms for scheduling divisible loads , 2005, IEEE Transactions on Parallel and Distributed Systems.

[21]  Rajkumar Buyya,et al.  A Linear Programming Driven Genetic Algorithm for Meta-Scheduling on Utility Grids , 2008, 2008 16th International Conference on Advanced Computing and Communications.

[22]  Victor V. Toporkov,et al.  Dependable Strategies for Job-Flows Dispatching and Scheduling in Virtual Organizations of Distributed Computing Environments , 2013 .

[23]  Ramin Yahyapour,et al.  Economic Scheduling in Grid Computing , 2002, JSSPP.

[24]  Victor V. Toporkov,et al.  Heuristic Co-allocation Strategies in Distributed Computing with Non-dedicated Resources , 2013, IDC.

[25]  Albert Y. Zomaya,et al.  Profit-driven scheduling for cloud services with data access awareness , 2012, J. Parallel Distributed Comput..

[26]  David Abramson,et al.  High performance parametric modeling with Nimrod/G: killer application for the global grid? , 2000, Proceedings 14th International Parallel and Distributed Processing Symposium. IPDPS 2000.

[27]  Victor V. Toporkov,et al.  JOB AND APPLICATION-LEVEL SCHEDULING IN DISTRIBUTED COMPUTING , 2009 .

[28]  David J. Pym,et al.  Economic aspects of a utility computing service , 2007, GridNets '07.

[29]  Cynthia Bailey Lee,et al.  Are User Runtime Estimates Inherently Inaccurate? , 2004, JSSPP.

[30]  Victor V. Toporkov,et al.  Job and Application-Level Scheduling: An Integrated Approach for Achieving Quality of Service in Distributed Computing , 2009, 2009 Fourth International Conference on Dependability of Computer Systems.

[31]  Yoshio Tanaka,et al.  An Advance Reservation-Based Co-allocation Algorithm for Distributed Computers and Network Bandwidth on QoS-Guaranteed Grids , 2010, JSSPP.