Path Optimization in Stream-Based Overlay Networks

The emergence of sensor networks and distributed applications that generate data streams has created a need for Internet overlays designed for streaming data. Such stream-based overlay network (SBONs) consist of a set of Internet hosts that collect, process, and deliver stream-based data to multiple applications. A key challenge in the design and implementation of SBONs is efficient path optimization when mapping logical query streams to physical network hosts and paths. Suboptimal placements can induce poor utilization of network resources, leading to severe performance penalties, link saturation, and network hotspots. Our goal is to realize efficient stream placement that takes the physical topology of the Internet into account, thereby minimizing overall network utilization. In this paper, we describe a novel, network-aware path optimization algorithm for stream-based overlay networks. Our approach is based on a spring relaxation model that operates in a metric space defined by the pairwise node latency within the SBON. This relaxation placement algorithm utilizes the underlying network resources efficiently, is capable of performing inter-stream optimization, and can be implemented in a scalable and decentralized way. To evaluate its performance, we define a set of evaluation metrics for path optimization in terms of network utilization, application delay, and resource contention. Our simulation experiments with a realistic network topology show that relaxation placement approaches optimal network utilization without introducing an undue delay penalty or resource contention. Compared to an optimal placement strategy, relaxation placement causes only 15% more network traffic on average, while adding a 24% delay penalty for the application. We validate our simulation results with actual measurements involving 70 PlanetLab nodes.

[1]  An Evaluation of Scalable Application-Level Multicast Built Using Peer-To-Peer Overlay Networks , 2002 .

[2]  Srinivasan Seshan,et al.  Cache-and-query for wide area sensor databases , 2003, SIGMOD '03.

[3]  Alex C. Snoeren,et al.  Mesh-based content routing using XML , 2001, SOSP.

[4]  Ellen W. Zegura,et al.  How to model an internetwork , 1996, Proceedings of IEEE INFOCOM '96. Conference on Computer Communications.

[5]  Ying Xing,et al.  Scalable Distributed Stream Processing , 2003, CIDR.

[6]  David Wetherall,et al.  Scriptroute: A Public Internet Measurement Facility , 2003, USENIX Symposium on Internet Technologies and Systems.

[7]  Guy M. Lohman,et al.  Optimizer Validation and Performance Evaluation for Distributed Queries , 1998 .

[8]  Joseph M. Hellerstein,et al.  Flux: an adaptive partitioning operator for continuous query systems , 2003, Proceedings 19th International Conference on Data Engineering (Cat. No.03CH37405).

[9]  Suman Banerjee,et al.  The Interdomain Connectivity of PlanetLab Nodes , 2004, PAM.

[10]  Frederick Reiss,et al.  TelegraphCQ: Continuous Dataflow Processing for an Uncertain World , 2003, CIDR.

[11]  Srinivasan Seshan,et al.  A case for end system multicast , 2002, IEEE J. Sel. Areas Commun..

[12]  StonebrakerMichael,et al.  Aurora: a new model and architecture for data stream management , 2003, VLDB 2003.

[13]  Michael Stonebraker,et al.  Mariposa: a wide-area distributed database system , 1996, The VLDB Journal.

[14]  Srinivasan Seshan,et al.  IrisNet: An Architecture for a Worldwide Sensor Web , 2003, IEEE Pervasive Comput..

[15]  Miguel Castro,et al.  Scribe: a large-scale and decentralized application-level multicast infrastructure , 2002, IEEE J. Sel. Areas Commun..

[16]  Liang Chen,et al.  GATES: a grid-based middleware for processing distributed data streams , 2004, Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004..

[17]  G. Weikum Querying the Internet with PIER , 2005 .

[18]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[19]  Kirk L. Johnson,et al.  Overcast: reliable multicasting with on overlay network , 2000, OSDI.

[20]  Ying Xing,et al.  The Design of the Borealis Stream Processing Engine , 2005, CIDR.

[21]  Jeffrey F. Naughton,et al.  Rate-based query optimization for streaming information sources , 2002, SIGMOD '02.

[22]  Margo I. Seltzer,et al.  Open problems in data collection networks , 2004, EW 11.

[23]  Peter R. Pietzuch,et al.  Composite event detection as a generic middleware extension , 2004, IEEE Network.

[24]  Donald Kossmann,et al.  The state of the art in distributed query processing , 2000, CSUR.

[25]  Hui Zhang,et al.  Predicting Internet network distance with coordinates-based approaches , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[26]  Robert Tappan Morris,et al.  Vivaldi: a decentralized network coordinate system , 2004, SIGCOMM '04.

[27]  Jon Crowcroft,et al.  Lighthouses for Scalable Distributed Location , 2003, IPTPS.

[28]  Robert Tappan Morris,et al.  Practical, distributed network coordinates , 2004, Comput. Commun. Rev..

[29]  Michael Stonebraker,et al.  Contract-Based Load Management in Federated Distributed Systems , 2004, NSDI.