Computational Resource Exchanges for Distributed Resource Allocation

Appropriate abstractions, mechanisms, and policies for resource allocation is quickly emerging as the fundamental problem facing emerging computation and communication environments such as PlanetLab and the Grid. This paper explores the utility of one simple abstraction for global resource allocation with a number of appealing properties: a centralized auction that collects user descriptions of resource configurations and the values placed on these configurations. The task of the clearinghouse is to determine a set of winning bids and to assign appropriate subsets of global resources to individual users. One challenge with this model is the computational complexity associated with determining winners. To make the problem tractable, we propose appropriate bidding languages that constrain the type of bids that users can make, while maintaining required expressiveness. Computing optimal solutions to such auctions for scales of current interest (e.g., 1000 nodes) is intractable on current hardware, even given aggressive optimizations. Thus, we introduce a number of heuristics that appear to perform well in practice. Another challenge with auctions is the lag in clearing the auction and the uncertainty in whether resources will actually be acquired. We introduce a formulation for “Buy it Now” pricing to address some of these limitations.

[1]  William Vickrey,et al.  Counterspeculation, Auctions, And Competitive Sealed Tenders , 1961 .

[2]  Ivan E. Sutherland,et al.  A futures market in computer time , 1968, Commun. ACM.

[3]  E. H. Clarke Multipart pricing of public goods , 1971 .

[4]  Theodore Groves,et al.  Incentives in Teams , 1973 .

[5]  Tad Hogg,et al.  Spawn: A Distributed Computational Economy , 1992, IEEE Trans. Software Eng..

[6]  Deborah Estrin,et al.  Pricing in computer networks: motivation, formulation, and example , 1993, TNET.

[7]  William E. Weihl,et al.  Lottery scheduling: flexible proportional-share resource management , 1994, OSDI '94.

[8]  Hussein M. Abdel-Wahab,et al.  A Microeconomic Scheduler for Parallel Computers , 1995, JSSPP.

[9]  Deborah Estrin,et al.  Pricing in Computer Networks: Reshaping the Research Agenda , 2020, The Internet and Telecommunications Policy.

[10]  S. Clearwater Market-based control: a paradigm for distributed resource allocation , 1996 .

[11]  Ian T. Foster,et al.  Globus: a Metacomputing Infrastructure Toolkit , 1997, Int. J. High Perform. Comput. Appl..

[12]  Rajesh Raman,et al.  Matchmaking: distributed resource management for high throughput computing , 1998, Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244).

[13]  Frank Kelly,et al.  Rate control for communication networks: shadow prices, proportional fairness and stability , 1998, J. Oper. Res. Soc..

[14]  N. Nisan,et al.  The POPCORN market—an online market for computational resources , 1998, ICE '98.

[15]  Stephen Russell,et al.  Resource management in the Mungi single-address-space operating system , 1998 .

[16]  Andrew B. Whinston,et al.  The economics of network management , 1999, CACM.

[17]  Yoav Shoham,et al.  Truth revelation in approximately efficient combinatorial auctions , 2002, EC '99.

[18]  Derek McAuley,et al.  Congestion prices as feedback signals: an approach to QoS management , 2000, ACM SIGOPS European Workshop.

[19]  Noam Nisan,et al.  Bidding and allocation in combinatorial auctions , 2000, EC '00.

[20]  Craig Boutilier,et al.  Bidding Languages for Combinatorial Auctions , 2001, IJCAI.

[21]  Noam Nisan,et al.  On-Line Markets for Distributed Object Services: The MAJIC System , 2001, USITS.

[22]  David C. Parkes,et al.  Achieving Budget-Balance with Vickrey-Based Payment Schemes in Exchanges , 2001, IJCAI.

[23]  Mark J. Clement,et al.  Core Algorithms of the Maui Scheduler , 2001, JSSPP.

[24]  Ian T. Foster,et al.  The anatomy of the grid: enabling scalable virtual organizations , 2001, Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid.

[25]  Noam Nisan,et al.  An efficient approximate allocation algorithm for combinatorial auctions , 2001, EC '01.

[26]  David Levine,et al.  CABOB: A Fast Optimal Algorithm for Combinatorial Auctions , 2001, IJCAI.

[27]  Mike Hibler,et al.  An integrated experimental environment for distributed systems and networks , 2002, OPSR.

[28]  David Levine,et al.  Winner determination in combinatorial auction generalizations , 2002, AAMAS '02.

[29]  Hari Balakrishnan,et al.  Resilient overlay networks , 2001, SOSP.

[30]  Marianne Shaw,et al.  Scale and performance in the Denali isolation kernel , 2002, OSDI '02.

[31]  Hector Garcia-Molina,et al.  Peer-to-Peer Resource Trading in a Reliable Distributed System , 2002, IPTPS.

[32]  Amin Vahdat,et al.  Workload and Failure Characterization on a Large-Scale Federated Testbed , 2003 .

[33]  Amin Vahdat,et al.  SHARP: an architecture for secure resource peering , 2003, SOSP '03.

[34]  David C. Parkes,et al.  Strategyproof Computing: Systems Infrastructures for Self-Interested Parties , 2003 .

[35]  HarrisTim,et al.  Xen and the art of virtualization , 2003 .

[36]  Amin Vahdat,et al.  Bootstrapping a Distributed Computational Economy with Peer-to-Peer Bartering , 2003 .

[37]  D. Krych CALCULATION AND ANALYSIS OF NASH EQUILIBRIA OF VICKREY-BASED PAYMENT RULES FOR COMBINATORIAL EXCHANGES , 2003 .

[38]  Sven de Vries,et al.  Combinatorial Auctions: A Survey , 2003, INFORMS J. Comput..

[39]  David E. Culler,et al.  A blueprint for introducing disruptive technology into the Internet , 2003, CCRV.

[40]  Mohammad Taghi Hajiaghayi,et al.  Adaptive limited-supply online auctions , 2004, EC '04.

[41]  David E. Culler,et al.  Operating Systems Support for Planetary-Scale Network Services , 2004, NSDI.