Scalable and Efficient End-to-End Network Topology Inference

To construct an efficient overlay network, the information of underlay is important. We consider using end-to-end measurement tools such as traceroute to infer the underlay topology among a group of hosts. Previously, Max-Delta has been proposed to infer a highly accurate topology with a low number of traceroutes. However, Max-Delta relies on a central server to collect traceroute results and to select paths for hosts to traceroute. It is not scalable to large groups. In this paper, we investigate a distributed inference scheme to support scalable inference. In our scheme, each host joins an overlay tree before conducting traceroute. A host then independently selects paths for tracerouting and exchanges traceroute results with others through the overlay tree. As a result, each host can maintain a partially discovered topology. We have studied the key issue in the scheme, that is, how a low-diameter overlay tree can be constructed. Furthermore, we propose several techniques to reduce the measurement cost for topology inference. They include 1) integrating the Doubletree algorithm into our scheme to reduce measurement redundancy, 2) setting up a lookup table for routers to reduce traceroute size, and 3) conducting topology abstraction and reducing the computational frequency to reduce the computational overhead. As compared to the naive Max-Delta, our scheme is fully distributed and scalable. The computational loads for target selection are distributed to all the hosts instead of a single server. In addition, each host only communicates with a few other hosts. The consumption of edge bandwidth at a host is hence limited. We have done simulations on Internet-like topologies and conducted measurements on PlanetLab. The results show that the constructed tree has a low diameter and can support quick data exchange between hosts. Furthermore, the proposed improvements can efficiently reduce measurement redundancy, bandwidth consumption, and computational overhead.

[1]  kc claffy,et al.  Internet topology: connectivity of IP graphs , 2001, SPIE ITCom.

[2]  Jonathan S. Turner,et al.  Routing in overlay multicast networks , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[3]  Marcel Waldvogel,et al.  Dimensioning server access bandwidth and multicast routing in overlay networks , 2001, NOSSDAV '01.

[4]  Lixin Gao,et al.  On inferring and characterizing Internet routing policies , 2003, Journal of Communications and Networks.

[5]  Hari Balakrishnan,et al.  Resilient overlay networks , 2001, SOSP.

[6]  Bo Li,et al.  CoolStreaming/DONet: a data-driven overlay network for peer-to-peer live media streaming , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..

[7]  Robert Nowak,et al.  Internet tomography , 2002, IEEE Signal Process. Mag..

[8]  Shueng-Han Gary Chan,et al.  Improving the Efficiency of End-to-End Network Topology Inference , 2007, 2007 IEEE International Conference on Communications.

[9]  Michalis Faloutsos,et al.  On power-law relationships of the Internet topology , 1999, SIGCOMM '99.

[10]  Mark Crovella,et al.  Improved Algorithms for Network Topology Discovery , 2005, PAM.

[11]  Mark Crovella,et al.  Efficient algorithms for large-scale topology discovery , 2004, SIGMETRICS '05.

[12]  David Watson,et al.  Topology aware overlay networks , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..

[13]  Shueng-Han Gary Chan,et al.  A Distributed Approach to End-to-End Network Topology Inference , 2007, 2007 IEEE International Conference on Communications.

[14]  Ratul Mahajan,et al.  Measuring ISP topologies with Rocketfuel , 2004, IEEE/ACM Transactions on Networking.

[15]  Fangzhe Chang,et al.  Topology inference in the presence of anonymous routers , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[16]  Mark Crovella,et al.  Deployment of an Algorithm for Large-Scale Topology Discovery , 2006, IEEE Journal on Selected Areas in Communications.

[17]  Sonia Fahmy,et al.  Topology-aware overlay networks for group communication , 2002, NOSSDAV '02.

[18]  Hui Zhang,et al.  Predicting Internet network distance with coordinates-based approaches , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[19]  Robert Tappan Morris,et al.  Vivaldi: a decentralized network coordinate system , 2004, SIGCOMM '04.

[20]  Zongming Fei,et al.  A Proactive Tree Recovery Mechanism for Resilient Overlay Multicast , 2007, IEEE/ACM Transactions on Networking.

[21]  Srinivasan Seshan,et al.  A case for end system multicast , 2002, IEEE J. Sel. Areas Commun..

[22]  Yajun Wang,et al.  Fast overlay tree based on efficient end-to-end measurements , 2005, IEEE International Conference on Communications, 2005. ICC 2005. 2005.

[23]  Bruce M. Maggs,et al.  The feasibility of supporting large-scale live streaming applications with dynamic application end-points , 2004, SIGCOMM.

[24]  Azer Bestavros,et al.  On the marginal utility of network topology measurements , 2001, IMW '01.

[25]  Nick Feamster,et al.  Topology inference from BGP routing dynamics , 2002, IMW '02.

[26]  Stephen A. Jarvis,et al.  Improving the Fault Resilience of Overlay Multicast for Media Streaming , 2006, IEEE Transactions on Parallel and Distributed Systems.

[27]  Shueng-Han Gary Chan,et al.  A Cost-Based Evaluation of End-to-End Network Measurements in Overlay Multicast , 2007, IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications.

[28]  Sriram Ramabhadran,et al.  A case study in building layered DHT applications , 2005, SIGCOMM '05.

[29]  Brighten Godfrey,et al.  OpenDHT: a public DHT service and its uses , 2005, SIGCOMM '05.

[30]  Ellen W. Zegura,et al.  How to model an internetwork , 1996, Proceedings of IEEE INFOCOM '96. Conference on Computer Communications.

[31]  Robert D. Nowak,et al.  Maximum likelihood network topology identification from edge-based unicast measurements , 2002, SIGMETRICS '02.

[32]  Yajun Wang,et al.  Network Topology Inference Based on End-to-End Measurements , 2006, IEEE Journal on Selected Areas in Communications.

[33]  Bobby Bhattacharjee,et al.  Scalable application layer multicast , 2002, SIGCOMM '02.

[34]  Ramesh Govindan,et al.  Heuristics for Internet map discovery , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[35]  Kirk L. Johnson,et al.  Overcast: reliable multicasting with on overlay network , 2000, OSDI.

[36]  Lakshminarayanan Subramanian,et al.  An investigation of geographic mapping techniques for internet hosts , 2001, SIGCOMM.