NETEMBED: A network resource mapping service for distributed applications

Emerging configurable infrastructures (large-scale overlays, grids, distributed testbeds, and sensor networks among others) comprise diverse sets of computing resources and network conditions. The distributed applications to be deployed on these infrastructures exhibit increasingly complex constraints and requirements on the resources they require. Thus, a common problem facing the efficient deployment of distributed applications on these infrastructures is that of mapping application-level requirements onto the network in such a manner that the requirements of the application are realized. We present two new techniques to tackle this combinatorially-hard problem that thanks to a number of heuristics, are able to find feasible solutions or determine the non-existence of a solution in most cases, where otherwise the problem would be intractable. These techniques are also false negative free, a common problem among other heuristics currently in use.

[1]  Robert Ricci,et al.  A solver for the network testbed mapping problem , 2003, CCRV.

[2]  Rajesh Raman,et al.  Matchmaking: distributed resource management for high throughput computing , 1998, Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244).

[3]  Robert Tappan Morris,et al.  Vivaldi: a decentralized network coordinate system , 2004, SIGCOMM '04.

[4]  Stephen Beale,et al.  HUNTER GATHERER Applying Constraint Satisfaction Branch and Bound and Solution Synthesis to Computational Semantics , 1997 .

[5]  Jay Lepreau,et al.  Lowering the barrier to wireless and mobile experimentation , 2003, CCRV.

[6]  James Robertson,et al.  UsenetDHT: A Low Overhead Usenet Server , 2004, IPTPS.

[7]  David E. Culler,et al.  A blueprint for introducing disruptive technology into the Internet , 2003, CCRV.

[8]  Anukool Lakhina,et al.  BRITE: Universal Topology Generation from a User''s Perspective , 2001 .

[9]  Geoffrey C. Fox,et al.  Matchmaking scientific workflows in grid environments , 2007 .

[10]  Jeffrey Considine,et al.  Informed content delivery across adaptive overlay networks , 2002, IEEE/ACM Transactions on Networking.

[11]  Daniel Zappala,et al.  Cluster Computing on the Fly : P 2 P Scheduling of Idle Cycles in the Internet , 2004 .

[12]  Aravind Srinivasan,et al.  Resilient multicast using overlays , 2003, IEEE/ACM Transactions on Networking.

[13]  Robert Ricci,et al.  Lessons from resource allocators for large-scale multiuser testbeds , 2006, OPSR.

[14]  Andrew A. Chien,et al.  Efficient resource description and high quality selection for virtual grids , 2005, CCGrid 2005. IEEE International Symposium on Cluster Computing and the Grid, 2005..

[15]  John W. Byers,et al.  A constraint satisfaction approach to testbed embedding services , 2004, Comput. Commun. Rev..

[16]  Amin Vahdat,et al.  Bullet: high bandwidth data dissemination using an overlay mesh , 2003, SOSP '03.

[17]  Miguel Castro,et al.  SplitStream: High-Bandwidth Content Distribution in Cooperative Environments , 2003, IPTPS.

[18]  Peter J. Stuckey,et al.  Programming with Constraints: An Introduction , 1998 .

[19]  Chuang Liu,et al.  A constraint language approach to matchmaking , 2004, 14th International Workshop Research Issues on Data Engineering: Web Services for e-Commerce and e-Government Applications, 2004. Proceedings..

[20]  Ellen W. Zegura,et al.  How to model an internetwork , 1996, Proceedings of IEEE INFOCOM '96. Conference on Computer Communications.

[21]  Yong Zhu,et al.  Algorithms for Assigning Substrate Network Resources to Virtual Network Components , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[22]  Paul Brett,et al.  A Shared Global Event Propagation System to Enable Next Generation Distributed Services , 2004, WORLDS.

[23]  Amin Vahdat,et al.  Design and implementation tradeoffs for wide-area resource discovery , 2005, HPDC-14. Proceedings. 14th IEEE International Symposium on High Performance Distributed Computing, 2005..

[24]  Azer Bestavros,et al.  snBench: programming and virtualization framework for distributed multitasking sensor networks , 2006, VEE '06.

[25]  U. Brandes,et al.  GraphML Progress Report ? Structural Layer Proposal , 2001 .

[26]  Rizos Sakellariou,et al.  A hybrid heuristic for DAG scheduling on heterogeneous systems , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[27]  Ken Kennedy,et al.  Scheduling strategies for mapping application workflows onto the grid , 2005, HPDC-14. Proceedings. 14th IEEE International Symposium on High Performance Distributed Computing, 2005..

[28]  Ben Y. Zhao,et al.  Maintenance-Free Global Data Storage , 2001, IEEE Internet Comput..

[29]  John W. Byers,et al.  ROMA: reliable overlay multicast with loosely coupled TCP connections , 2004, IEEE INFOCOM 2004.

[30]  David E. Culler,et al.  The ganglia distributed monitoring system: design, implementation, and experience , 2004, Parallel Comput..

[31]  William Gropp,et al.  MPICH-GQ: Quality-of-Service for Message Passing Programs , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[32]  Srinivasan Seshan,et al.  A case for end system multicast , 2002, IEEE J. Sel. Areas Commun..

[33]  KyoungSoo Park,et al.  CoMon: a mostly-scalable monitoring system for PlanetLab , 2006, OPSR.

[34]  Rajesh Raman,et al.  Policy driven heterogeneous resource co-allocation with Gangmatching , 2003, High Performance Distributed Computing, 2003. Proceedings. 12th IEEE International Symposium on.

[35]  Mike Hibler,et al.  An integrated experimental environment for distributed systems and networks , 2002, OSDI '02.