Resource allocation in the grid using reinforcement learning

In this paper we study a minimalist decentralized algorithm for resource allocation in a simplified Grid-like environment. We consider a system consisting of large number of heterogenous reinforcement learning agents that share common resources for their computational needs. There is no communication between the agents: the only information that agents receive is the (expected) completion time of a job it submitted to a particular resource and which serves as a reinforcement signal for the agent. The results of our experiments suggest that reinforcement learning can be used to improve the quality of resource allocation in large scale heterogenous system.

[1]  Rajesh Raman,et al.  Resource management through multilateral matchmaking , 2000, Proceedings the Ninth International Symposium on High-Performance Distributed Computing.

[2]  Kristina Lerman,et al.  Resource allocation games with changing resource capacities , 2003, AAMAS '03.

[3]  Daniel Kudenko,et al.  Reinforcement learning of coordination in cooperative multi-agent systems , 2002, AAAI/IAAI.

[4]  Yi-Cheng Zhang,et al.  Emergence of cooperation and organization in an evolutionary game , 1997 .

[5]  Kavitha Ranganathan,et al.  Decoupling computation and data scheduling in distributed data-intensive applications , 2002, Proceedings 11th IEEE International Symposium on High Performance Distributed Computing.

[6]  N. Johnson,et al.  Minority game with arbitrary cutoffs , 1999, cond-mat/9903228.

[7]  R. Rosenthal.  A class of games possessing pure-strategy Nash equilibria , 1973 .

[8]  Rajesh Raman,et al.  Matchmaking: distributed resource management for high throughput computing , 1998, Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244).

[9]  Ian T. Foster,et al.  Grid information services for distributed resource sharing , 2001, Proceedings 10th IEEE International Symposium on High Performance Distributed Computing.

[10]  Craig Boutilier,et al.  The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems , 1998, AAAI/IAAI.

[11]  Ian T. Foster,et al.  The anatomy of the grid: enabling scalable virtual organizations , 2001, Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid.

[12]  Richard Wolski,et al.  Forecasting network performance to support dynamic scheduling using the network weather service , 1997, Proceedings. The Sixth IEEE International Symposium on High Performance Distributed Computing (Cat. No.97TB100183).

[13]  M Marsili,et al.  Phase transition and symmetry breaking in the minority game. , 1999, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[14]  Warren Smith,et al.  A Resource Management Architecture for Metacomputing Systems , 1998, JSSPP.

[15]  J. Crutchfield,et al.  Coupled replicator equations for the dynamics of learning in multiagent systems. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[16]  Yolanda Gil,et al.  Pegasus: Mapping Scientific Workflows onto the Grid , 2004, European Across Grids Conference.

[17]  Yoav Shoham,et al.  Multi-Agent Reinforcement Learning:a critical survey , 2003 .

[18]  Adam Arbree,et al.  Mapping Abstract Complex Workflows onto Grid Environments , 2003, Journal of Grid Computing.

[19]  Ian T. Foster,et al.  SNAP: A Protocol for Negotiating Service Level Agreements and Coordinating Resource Management in Distributed Systems , 2002, JSSPP.

[20]  Robert Richards,et al.  Universal Description, Discovery, and Integration (UDDI) , 2006 .

[21]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[22]  Moshe Tennenholtz,et al.  Adaptive Load Balancing: A Study in Multi-Agent Learning , 1994, J. Artif. Intell. Res..

[23]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[24]  Kristina Lerman,et al.  Adaptive Boolean networks and minority games with time-dependent capacities. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[25]  B. Barish,et al.  LIGO and the Detection of Gravitational Waves , 1999 .

[26]  Gregor von Laszewski,et al.  A fault detection service for wide area distributed computations , 2004, Cluster Computing.

[27]  W. Arthur.  Inductive Reasoning and Bounded Rationality , 1994 .