Efficient Information Dissemination in Wide Area Heterogeneous Overlay Networks

In this dissertation research we study and address the unique challenges involved in information sharing and dissemination of large-scale group communication applications. We focus on system architectures and various techniques for efficient and scalable information dissemination in distributed P2P environments. Our solutions are developed by targeting at utilizing three representative P2P overlay networks: structured P2P network based on consistent hashing techniques, unstructured Gnutella-like P2P network, and P2P GeoGrid based on geographical location and proximity of end nodes. We have made three unique contributions to the general field of large-scale information sharing and dissemination. First, we propose a landmark-based peer clustering techniques to grouping end-system nodes by their network proximity, and a communication management technique addresses load balancing and reliability of group communication applications in structured P2P network. Second, we develop a utility-based P2P group communication service middleware, consisting of a utility-based topology management and a utility-aware P2P routing, for providing scalable and efficient group communication services in an unstructured P2P overlay network of heterogeneous peers. Third, we propose an overlay network management protocol that is aware of the geographical location of end-system nodes and a set of routing and adaptation techniques, aiming at building decentralized information dissemination service networks to support location-based applications and services. Although different overlay networks require different system designs for building scalable and efficient information dissemination services, we have employed two common design philosophies: (1) exploiting end-system heterogeneity and (2) utilizing proximity information of end-system nodes to localize most of the communication traffic, and (3) using randomized shortcuts to accelerate long-distant communications. We have demonstrated our design philosophies and the performance improvements in the above three types of P2P overlay networks. Concretely, by assigning more workloads to more powerful peers, we can greatly increase the system scalability and reduce the variation of workload distribution. By clustering end-system nodes based on their IP-network proximity or their geographical proximity, and utilizing randomized shortcuts, we can reduce the end-to-end communication latency, balance peer workloads against service request hotspots across the overlay network, and significantly enhance the scalability and efficiency of large-scale decentralized information dissemination and group communication.