Tycoon: An implementation of a distributed, market-based resource allocation system

Distributed clusters like the Grid and PlanetLab enable the same statistical multiplexing efficiency gains for computing as the Internet provides for networking. One major challenge is allocating resources in an economically efficient and low-latency way. A common solution is proportional share, where users each get resources in proportion to their pre-defined weight. However, this does not allow users to differentiate the value of their jobs. This leads to economic inefficiency. In contrast, systems that require reservations impose a high latency (typically minutes to hours) to acquire resources. This paper describes Tycoon, a market based distributed resource allocation system based on proportional share. The key advantages of Tycoon are that it allows users to differentiate the values of their jobs, its resource acquisition latency is limited only by communication delays, and it imposes no manual bidding overhead on users. Experimental results using a prototype implementation of the design are included.

[1]  L. Hurwicz.  The Design of Mechanisms for Resource Allocation , 1973 .

[2]  Jerome H. Saltzer,et al.  End-to-end arguments in system design , 1984, TOCS.

[3]  Tad Hogg,et al.  Spawn: A Distributed Computational Economy , 1992, IEEE Trans. Software Eng..

[4]  William E. Weihl,et al.  Lottery scheduling: flexible proportional-share resource management , 1994, OSDI '94.

[5]  Hussein M. Abdel-Wahab,et al.  A Microeconomic Scheduler for Parallel Computers , 1995, JSSPP.

[6]  Hal R. Varian,et al.  Economic Mechanism Design for Computerized Agents , 1995, USENIX Workshop on Electronic Commerce.

[7]  David D. Clark,et al.  The design philosophy of the DARPA internet protocols , 1988, SIGCOMM '88.

[8]  Rudolf Schneider,et al.  On the chairman assignment problem , 1996, Discret. Math..

[9]  Hussein M. Abdel-Wahab,et al.  A proportional share resource allocation algorithm for real-time, time-shared systems , 1996, 17th IEEE Real-Time Systems Symposium.

[10]  Scott Devine,et al.  Disco: running commodity operating systems on scalable multiprocessors , 1997, TOCS.

[11]  Ian T. Foster,et al.  Globus: a Metacomputing Infrastructure Toolkit , 1997, Int. J. High Perform. Comput. Appl..

[12]  M. R. Rao,et al.  Combinatorial Optimization , 1992, NATO ASI Series.

[13]  William J. Cook,et al.  Combinatorial optimization , 1997 .

[14]  Margo I. Seltzer,et al.  Isolation with Flexibility: A Resource Management Framework for Central Servers , 2000, USENIX Annual Technical Conference, General Track.

[15]  David E. Culler,et al.  Market-based Proportional Resource Sharing for Clusters , 2000 .

[16]  Michael P. Wellman,et al.  Auction Protocols for Decentralized Scheduling , 2001, Games Econ. Behav..

[17]  H. Liu,et al.  Conference on Measurement and modeling of computer systems , 2001 .

[18]  Jason Nieh,et al.  Virtual-Time Round-Robin: An O(1) Proportional Share Scheduler , 2001, USENIX Annual Technical Conference, General Track.

[19]  Prashant J. Shenoy,et al.  Resource overbooking and application profiling in shared hosting platforms , 2002, OSDI '02.

[20]  Amin Vahdat,et al.  Bullet: high bandwidth data dissemination using an overlay mesh , 2003, SOSP '03.

[21]  Andrew Warfield,et al.  Xen and the art of virtualization , 2003, SOSP '03.

[22]  Amin Vahdat,et al.  SHARP: an architecture for secure resource peering , 2003, SOSP '03.

[23]  HarrisTim,et al.  Xen and the art of virtualization , 2003 .

[24]  Adam Wierman,et al.  Classifying scheduling policies with respect to unfairness in an M/GI/1 , 2003, SIGMETRICS '03.

[25]  David E. Culler,et al.  A blueprint for introducing disruptive technology into the Internet , 2003, CCRV.

[26]  Bernardo A. Huberman,et al.  Tycoon: A Market-Based Resource Allocation System , 2004, ArXiv.

[27]  Mohammad Taghi Hajiaghayi,et al.  Adaptive limited-supply online auctions , 2004, EC '04.

[28]  Jeannie R. Albrecht,et al.  Computational Resource Exchanges for Distributed Resource Allocation , 2004 .

[29]  David E. Culler,et al.  Operating Systems Support for Planetary-Scale Network Services , 2004, NSDI.

[30]  Amin Vahdat,et al.  Resource Allocation in Federated Distributed Computing Infrastructures , 2004 .

[31]  Amin Vahdat,et al.  Scalable Wide-Area Resource Discovery , 2004 .

[32]  David E. Culler,et al.  The ganglia distributed monitoring system: design, implementation, and experience , 2004, Parallel Comput..

[33]  Vinton G. Cerf,et al.  A protocol for packet network intercommunication , 1974, CCRV.