Analyzing Cross-layer Interaction in Overlay Networks

Overlay networks have recently gained popularity as a viable alternative to overcome functionality limitations of the Internet (e.g., lack of QoS, multicast routing). They offer enhanced functionality to end-users by forming an independent and customizable virtual network over the native network. Furthermore, they are being widely promoted as a potential architecture of the future Internet in the form of network virtualization, where multiple heterogeneous virtual networks may co-exist on top of a shared native network. The prominent characteristic in either context is that routing at the overlay layer operates independent of that at the underlying native layer. There are several potential problems with this approach because overlay networks are selfish entities that are chiefly concerned with achieving the routing objective of their own users. This leads to complex cross-layer interactions between the native and overlay layers, and often tends to degrade the achieved performance for both layers. As overlay applications proliferate and the amount of selfish overlay traffic surges, there is a clear need for understanding the complex interactions and for strategies to manage them appropriately. Our work addresses these issues in the context of service overlay networks, which represent virtual networks formed of persistent nodes that collaborate to offer improved services to actual end-systems. Typically, service overlays alter the route between the overlay nodes in a dynamic manner in order to satisfy a selfish objective. The objective of this thesis is to improve the stability and performance of overlay routing in this multi-layer environment. We investigate the common problems of functionality overlap, lack of cross-layer awareness, mismatch or misalignment in routing objectives and the contention for native resources between the two layers. These problems often lead to deterioration in performance for the end-users. This thesis presents an analysis of the cross-layer interaction during fault recovery, inter-domain policy enforcement and traffic engineering in the multi-layer context. Based on our characterization of the interaction, we propose effective strategies that improve overall routing performance, with minimal side-effects on other traffic. These strategies typically (1) increase the layer-awareness (awareness of information about the other layer) at each layer, (2) introduce better control over routing dynamics and (3) offer improved overlay node placement options. Our results demonstrate how applying these strategies lead to better management of the cross-layer interaction, which in turn leads to improved routing performance for end-users.

[1]  David E. Culler,et al.  A blueprint for introducing disruptive technology into the Internet , 2003, CCRV.

[2]  Arthur Warburton,et al.  Approximation of Pareto Optima in Multiple-Objective, Shortest-Path Problems , 1987, Oper. Res..

[3]  Joseph D. Touch,et al.  Dynamic Internet overlay deployment and management using the X-Bone , 2000, Proceedings 2000 International Conference on Network Protocols.

[4]  Klara Nahrstedt,et al.  An overview of quality of service routing for next-generation high-speed networks: problems and solutions , 1998, IEEE Netw..

[5]  Piet Demeester,et al.  Network Recovery: Protection and Restoration of Optical, SONET-SDH, IP, and MPLS , 2004 .

[6]  Chuan Wu,et al.  Strategies of Conflict in Coexisting Streaming Overlays , 2007, IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications.

[7]  Yong Zhu,et al.  Algorithms for Assigning Substrate Network Resources to Virtual Network Components , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[8]  Ellen W. Zegura,et al.  Optimizing End-to-End Throughput for Data Transfers on an Overlay-TCP Path , 2005, NETWORKING.

[9]  Scott Shenker,et al.  Overcoming the Internet impasse through virtualization , 2005, Computer.

[10]  Yiwei Thomas Hou,et al.  Service Oriented Internet , 2003, ICSOC.

[11]  Ratul Mahajan,et al.  Understanding BGP misconfiguration , 2002, SIGCOMM 2002.

[12]  Randy H. Katz,et al.  On failure detection algorithms in overlay networks , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..

[13]  Mark Handley,et al.  TCP Congestion Window Validation , 2000, RFC.

[14]  Larry L. Peterson,et al.  An overlay data plane for PlanetLab , 2005, Advanced Industrial Conference on Telecommunications/Service Assurance with Partial and Intermittent Resources Conference/E-Learning on Telecommunications Workshop (AICT/SAPIR/ELETE'05).

[15]  Arun Venkataramani,et al.  Do Incentives Build Robustness in BitTorrent? (Awarded Best Student Paper) , 2007, NSDI.

[16]  Murali S. Kodialam,et al.  Dynamic routing of bandwidth guaranteed tunnels with restoration , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[17]  Chip Elliott,et al.  GENI - global environment for network innovations , 2008, LCN.

[18]  H. W. Corley,et al.  Shortest paths in networks with vector weights , 1985 .

[19]  Ratul Mahajan,et al.  Measuring ISP topologies with rocketfuel , 2002, TNET.

[20]  Johan A. Pouwelse,et al.  The Bittorrent P2P File-Sharing System: Measurements and Analysis , 2005, IPTPS.

[21]  Jonathan S. Turner,et al.  Efficient Mapping of Virtual Networks onto a Shared Substrate , 2006 .

[22]  Amin Vahdat,et al.  Opus: an overlay peer utility service , 2002, 2002 IEEE Open Architectures and Network Programming Proceedings. OPENARCH 2002 (Cat. No.02EX571).

[23]  Mostafa H. Ammar,et al.  Preemptive Strategies to Improve Routing Performance of Native and Overlay Layers , 2007, IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications.

[24]  Christos H. Papadimitriou,et al.  Worst-case equilibria , 1999 .

[25]  Ben Y. Zhao,et al.  Brocade: Landmark Routing on Overlay Networks , 2002, IPTPS.

[26]  Christos Gkantsidis,et al.  Network coding for large scale content distribution , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..

[27]  B. Cohen,et al.  Incentives Build Robustness in Bit-Torrent , 2003 .

[28]  Donald F. Towsley,et al.  Can an Overlay Compensate for a Careless Underlay? , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[29]  Henning Schulzrinne,et al.  An Analysis of the Skype Peer-to-Peer Internet Telephony Protocol , 2004, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[30]  Kirk L. Johnson,et al.  Overcast: reliable multicasting with on overlay network , 2000, OSDI.

[31]  W. Norton A Business Case for ISP Peering , 2002 .

[32]  Mohamed Ali Kâafar,et al.  Towards a Two-Tier Internet Coordinate System to Mitigate the Impact of Triangle Inequality Violations , 2008, Networking.

[33]  Xenofontas A. Dimitropoulos,et al.  Creating realistic BGP models , 2003, 11th IEEE/ACM International Symposium on Modeling, Analysis and Simulation of Computer Telecommunications Systems, 2003. MASCOTS 2003..

[34]  Pablo Rodriguez,et al.  Should internet service providers fear peer-assisted content distribution? , 2005, IMC '05.

[35]  Gordon T. Wilfong,et al.  Analysis of the MED oscillation problem in BGP , 2002, 10th IEEE International Conference on Network Protocols, 2002. Proceedings..

[36]  Ram Keralapura,et al.  Can ISPs Take the Heat from Overlay Networks ? , 2004 .

[37]  Yin Zhang,et al.  On AS-level path inference , 2005, SIGMETRICS '05.

[38]  Vern Paxson,et al.  End-to-end Internet packet dynamics , 1997, SIGCOMM '97.

[39]  Mostafa H. Ammar,et al.  On the Interaction Between Dynamic Routing in Native and Overlay Layers , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[40]  Matthias Ehrgott,et al.  Multicriteria Optimization , 2005 .

[41]  Leslie G. Valiant,et al.  A Scheme for Fast Parallel Communication , 1982, SIAM J. Comput..

[42]  S. Seetharaman,et al.  Overlay-Friendly Native Network : A Contradiction in Terms ? , 2005 .

[43]  C. Siva Ram Murthy,et al.  Efficient algorithms for routing dependable connections in WDM optical networks , 2001, TNET.

[44]  Pavlin Radoslavov,et al.  Designing extensible IP router software , 2005, NSDI.

[45]  Mor Harchol-Balter,et al.  Evaluation of Task Assignment Policies for Supercomputing Servers: The Case for Load Unbalancing and Fairness , 2004, Cluster Computing.

[46]  Yong Zhu,et al.  Dynamic overlay routing based on available bandwidth estimation: A simulation study , 2006, Comput. Networks.

[47]  Yakov Rekhter,et al.  Mpls: Technology and Applications , 2000 .

[48]  Donald F. Towsley,et al.  Modeling TCP throughput: a simple model and its empirical validation , 1998, SIGCOMM '98.

[49]  Joseph D. Touch,et al.  A Recursive Network Architecture , 2006 .

[50]  Christos Gkantsidis,et al.  Anatomy of a P2P Content Distribution system with Network Coding , 2006, IPTPS.

[51]  Don Towsley,et al.  On the Interaction Between Overlay Routing and Traffic Engineering , 2004 .

[52]  J. Moy,et al.  OSPF: Anatomy of an Internet Routing Protocol , 1998 .

[53]  Lorne Mason,et al.  Restoration strategies and spare capacity requirements in self-healing ATM networks , 1997, Proceedings of INFOCOM '97.

[54]  Yin Zhang,et al.  On selfish routing in internet-like environments , 2006, TNET.

[55]  Manish Jain,et al.  Pathload: A Measurement Tool for End-to-End Available Bandwidth , 2002 .

[56]  Gustavo de Veciana,et al.  Service capacity of peer to peer networks , 2004, IEEE INFOCOM 2004.

[57]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[58]  Jerome H. Saltzer,et al.  End-to-end arguments in system design , 1984, TOCS.

[59]  Scott Shenker,et al.  Internet indirection infrastructure , 2002, SIGCOMM 2002.

[60]  Yiwei Thomas Hou,et al.  Service overlay networks: SLAs, QoS, and bandwidth provisioning , 2003, TNET.

[61]  Tim Roughgarden,et al.  How bad is selfish routing? , 2002, JACM.

[62]  Robert Tappan Morris,et al.  TCP behavior with many flows , 1997, Proceedings 1997 International Conference on Network Protocols.

[63]  Michalis Faloutsos,et al.  BLINC: multilevel traffic classification in the dark , 2005, SIGCOMM '05.

[64]  Mikel Izal,et al.  Dissecting BitTorrent: Five Months in a Torrent's Lifetime , 2004, PAM.

[65]  Prasant Mohapatra,et al.  The impact of topology on overlay routing service , 2004, IEEE INFOCOM 2004.

[66]  Lixin Gao,et al.  On the evaluation of AS relationship inferences [Internet reachability/traffic flow applications] , 2004, IEEE Global Telecommunications Conference, 2004. GLOBECOM '04..

[67]  Biswanath Mukherjee,et al.  Fault management in IP-over-WDM networks: WDM protection versus IP restoration , 2002, IEEE J. Sel. Areas Commun..

[68]  Venkata N. Padmanabhan,et al.  Some observations on bitTorrent performance , 2005, SIGMETRICS '05.

[69]  Sujata Banerjee,et al.  S3: a scalable sensing service for monitoring large networked systems , 2006, INM '06.

[70]  Yakov Rekhter,et al.  A Border Gateway Protocol 4 (BGP-4) , 1994, RFC.

[71]  Amin Vahdat,et al.  Detour: a Case for Informed Internet Routing and Transport , 2007 .

[72]  Mark Allman,et al.  On making TCP more robust to packet reordering , 2002, CCRV.

[73]  Patrick Crowley,et al.  Virtualizing Network Processors , 2006 .

[74]  Christian Scheideler,et al.  Can ISPS and P2P users cooperate for improved performance? , 2007, CCRV.

[75]  Xiaoning Ding,et al.  Measurements, analysis, and modeling of BitTorrent-like systems , 2005, IMC '05.

[76]  Chen-Nee Chuah,et al.  Virtual Multi-Homing: On the Feasibility of Combining Overlay Routing with BGP Routing , 2005, NETWORKING.

[77]  Jonathan S. Turner,et al.  Placing Servers in Overlay Networks , 2002 .

[78]  Srinivasan Seshan,et al.  A case for end system multicast , 2002, IEEE J. Sel. Areas Commun..

[79]  David Watson,et al.  Topology aware overlay networks , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..

[80]  Timothy Roscoe,et al.  The End of Internet Architecture , 2006, HotNets.

[81]  R. Sami,et al.  The Growth of Internet Overlay Networks : Implications for Architecture , Industry Structure and Policy , 2005 .

[82]  Guillaume Urvoy-Keller,et al.  Rarest first and choke algorithms are enough , 2006, IMC '06.

[83]  A. Orda,et al.  Ieee/acm Transactions on Networking 1 Achieving Network Optima Using Stackelberg Routing Strategies , 1997 .

[84]  Lixin Gao,et al.  On inferring autonomous system relationships in the Internet , 2000, Globecom '00 - IEEE. Global Telecommunications Conference. Conference Record (Cat. No.00CH37137).

[85]  Christophe Diot,et al.  Traffic matrix estimation: existing techniques and new directions , 2002, SIGCOMM 2002.

[86]  Deborah Estrin,et al.  The impact of routing policy on Internet paths , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[87]  Wu-chi Feng,et al.  Achieving faster failure detection in OSPF networks , 2003, IEEE International Conference on Communications, 2003. ICC '03..

[88]  Luca Valcarenghi,et al.  IP restoration vs. WDM protection: is there an optimal choice? , 2000, IEEE Netw..

[89]  Donald F. Towsley,et al.  Characterizing and Detecting Skype-Relayed Traffic , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[90]  Randy H. Katz,et al.  Backup path allocation based on a correlated link failure probability model in overlay networks , 2002, 10th IEEE International Conference on Network Protocols, 2002. Proceedings..

[91]  Yatin Chawathe Scattercast: an adaptable broadcast distribution framework , 2003, Multimedia Systems.

[92]  Parameswaran Ramanathan,et al.  What do packet dispersion techniques measure? , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[93]  E. Martins,et al.  A bicriterion shortest path algorithm , 1982 .

[94]  Michael B. Jones,et al.  Overlook: scalable name service on an overlay network , 2002, Proceedings 22nd International Conference on Distributed Computing Systems.

[95]  Rayadurgam Srikant,et al.  Modeling and performance analysis of BitTorrent-like peer-to-peer networks , 2004, SIGCOMM 2004.

[96]  Venkata N. Padmanabhan,et al.  Analyzing and Improving a BitTorrent Networks Performance Mechanisms , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[97]  G.J. Minden,et al.  A survey of active network research , 1997, IEEE Communications Magazine.

[98]  Jonathan S. Turner,et al.  Diversifying the Internet , 2005, GLOBECOM '05. IEEE Global Telecommunications Conference, 2005..

[99]  Chen-Nee Chuah,et al.  Can coexisting overlays inadvertently step on each other? , 2005, 13TH IEEE International Conference on Network Protocols (ICNP'05).

[100]  Anees Shaikh,et al.  A comparison of overlay routing and multihoming route control , 2004, SIGCOMM 2004.

[101]  Hari Balakrishnan,et al.  Resilient overlay networks , 2001, SOSP.

[102]  Athina Markopoulou,et al.  Characterization of failures in an IP backbone , 2004, IEEE INFOCOM 2004.

[103]  E. Martins On a multicriteria shortest path problem , 1984 .

[104]  Cengiz Alaettinoglu,et al.  Routing Policy Specification Language (RPSL) , 1998, RFC.

[105]  William Chan,et al.  Improving Traffic Locality in BitTorrent via Biased Neighbor Selection , 2006, 26th IEEE International Conference on Distributed Computing Systems (ICDCS'06).

[106]  Dorit S. Hochbaum,et al.  Approximation Algorithms for NP-Hard Problems , 1996 .

[107]  Angelos D. Keromytis,et al.  SOS: secure overlay services , 2002, SIGCOMM '02.

[108]  Yin Zhang,et al.  COPE: traffic engineering in dynamic networks , 2006, SIGCOMM 2006.

[109]  Daniel Massey,et al.  Collecting the internet AS-level topology , 2005, CCRV.

[110]  David Wetherall,et al.  Scriptroute: a facility for distributed internet measurement , 2003 .

[111]  Piet Demeester,et al.  PANEL: Protection Across NEtwork Layers , 1997 .

[112]  Cheng Jin,et al.  MATE: MPLS adaptive traffic engineering , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[113]  Scott Poretsky Considerations for Benchmarking Link-State IGP Data Plane Route Convergence , 2009 .

[114]  Randy H. Katz,et al.  An algebraic approach to practical and scalable overlay network monitoring , 2004, SIGCOMM 2004.

[115]  J. Turner,et al.  Diversifying the Network Edge , 2005 .

[116]  Sally Floyd,et al.  Quick-Start for TCP and IP , 2007, RFC.

[117]  Mostafa H. Ammar,et al.  Characterizing and Mitigating Inter-domain Policy Violations in Overlay Routes , 2006, Proceedings of the 2006 IEEE International Conference on Network Protocols.

[118]  Srikanth Kandula,et al.  Walking the tightrope: responsive yet stable traffic engineering , 2005, SIGCOMM '05.

[119]  Vaduvur Bharghavan,et al.  A new multicasting-based architecture for Internet host mobility , 1997, MobiCom '97.

[120]  Arnaud Legout Understanding BitTorrent: An Experimental Perspective , 2005 .

[121]  Nick Feamster,et al.  In VINI veritas: realistic and controlled network experimentation , 2006, SIGCOMM.

[122]  Kenneth L. Calvert,et al.  Modeling Internet topology , 1997, IEEE Commun. Mag..

[123]  William Allen Simpson,et al.  IP in IP Tunneling , 1995, RFC.

[124]  Mikkel Thorup,et al.  Internet traffic engineering by optimizing OSPF weights , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[125]  Marc Despontin,et al.  Multiple Criteria Optimization: Theory, Computation, and Application, Ralph E. Steuer (Ed.). Wiley, Palo Alto, CA (1986) , 1987 .

[126]  Jia Wang,et al.  Scalable and accurate identification of AS-level forwarding paths , 2004, IEEE INFOCOM 2004.

[127]  Randy H. Katz,et al.  OverQoS: offering Internet QoS using overlays , 2003, CCRV.

[128]  Olivier Bonaventure,et al.  Interdomain traffic engineering with BGP , 2003, IEEE Commun. Mag..

[129]  Pierre Hansen,et al.  Bicriterion Path Problems , 1980 .

[130]  Joseph D. Touch,et al.  DynaBone: dynamic defense using multi-layer Internet overlays , 2003, Proceedings DARPA Information Survivability Conference and Exposition.

[131]  Abhijit Bose,et al.  Delayed Internet routing convergence , 2000, SIGCOMM.

[132]  Yong Zhu,et al.  Overlay Network Assignment in PlanetLab With NetFinder , 2006 .

[133]  Jia Wang,et al.  Analyzing peer-to-peer traffic across large networks , 2004, IEEE/ACM Trans. Netw..

[134]  Weibo Gong,et al.  Understanding the interaction between overlay routing and Traffic Engineering , 2004 .

[135]  L. Freeman,et al.  Centrality in valued graphs: A measure of betweenness based on network flow , 1991 .

[136]  David S. Johnson,et al.  Computers and In stractability: A Guide to the Theory of NP-Completeness. W. H Freeman, San Fran , 1979 .

[137]  John Jannotti,et al.  Network layer support for overlay networks , 2002, 2002 IEEE Open Architectures and Network Programming Proceedings. OPENARCH 2002 (Cat. No.02EX571).

[138]  Cengiz Alaettinoglu,et al.  Towards Milli-Second IGP Convergence , 2000 .

[139]  Akihiro Nakao,et al.  A routing underlay for overlay networks , 2003, SIGCOMM '03.