Capability-Aware Information Aggregation in Peer-to-Peer Grids

Information aggregation is the process of summarizing information across the nodes of a distributed system. We present a hierarchical information aggregation system tailored for Peer-to-Peer Grids which typically exhibit a high degree of volatility and heterogeneity of resources. Aggregation is performed in a scalable yet efficient way by merging data along the edges of a logical self-healing tree with each inner node providing a summary view of the information delivered by the nodes of the corresponding subtree. We describe different tree management methods suitable for high-efficiency and high-scalability scenarios that take host capability and stability diversity into account to attenuate the impact of slow and/or unstable hosts. We propose an architecture covering all three phases of the aggregation process: Data gathering through a highly extensible sensing framework, data aggregation using reusable, fully isolated reduction networks, and application-sensitive data delivery using a broad range of propagation strategies. Our solution combines the advantages of approaches based on Distributed Hash Tables (DHTs) (i.e., load balancing and self-maintenance) and hierarchical approaches (i.e., respecting administrative boundaries and resource limitations). Our approach is integrated into our Peer-to-Peer Grid platform Cohesion. We substantiate its effectiveness through performance measurements and demonstrate its applicability through a graphical monitoring solution leveraging our aggregation system.

[1]  Robbert van Renesse,et al.  Astrolabe: A robust and scalable technology for distributed system monitoring, management, and data mining , 2003, TOCS.

[2]  Kenneth P. Birman,et al.  Bimodal multicast , 1999, TOCS.

[3]  Kazuyuki Shudo,et al.  P3: P2P-based middleware enabling transfer and aggregation of computational resources , 2005, CCGrid 2005. IEEE International Symposium on Cluster Computing and the Grid, 2005..

[4]  Eliza Varney Distributed Management Task Force, Inc , 2010 .

[5]  Wolfgang Blochinger,et al.  COHESION - A microkernel based Desktop Grid platform for irregular task-parallel applications , 2008, Future Gener. Comput. Syst..

[6]  Anne-Marie Kermarrec,et al.  Peer-to-Peer Membership Management for Gossip-Based Protocols , 2003, IEEE Trans. Computers.

[7]  Wei Hong,et al.  Proceedings of the 5th Symposium on Operating Systems Design and Implementation Tag: a Tiny Aggregation Service for Ad-hoc Sensor Networks , 2022 .

[8]  Johannes Gehrke,et al.  Gossip-based computation of aggregate information , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[9]  Praveen Yalagandula,et al.  A scalable distributed information management system , 2004, SIGCOMM 2004.

[10]  Beatrice Gralton,et al.  Washington DC - USA , 2008 .

[11]  David P. Anderson,et al.  BOINC: a system for public-resource computing and storage , 2004, Fifth IEEE/ACM International Workshop on Grid Computing.

[12]  Márk Jelasity,et al.  Robust aggregation protocols for large-scale overlay networks , 2004, International Conference on Dependable Systems and Networks, 2004.

[13]  Doug Terry,et al.  Epidemic algorithms for replicated database maintenance , 1988, OPSR.

[14]  Tim Moors,et al.  Survey of research towards robust peer-to-peer networks: Search methods , 2006, Comput. Networks.

[15]  Gilles Fedak,et al.  The Computational and Storage Potential of Volunteer Computing , 2006, Sixth IEEE International Symposium on Cluster Computing and the Grid (CCGRID'06).

[16]  Robbert van Renesse The importance of aggregation , 2003 .

[17]  Greg Ruetsch,et al.  Framework for Peer-to-Peer Distributed Computing in a Heterogeneous, Decentralized Environment , 2002, GRID.

[18]  Andrew A. Chien,et al.  Henri Casanova , 2022 .

[19]  Ben Y. Zhao,et al.  Future Directions in Distributed Computing , 2003, Lecture Notes in Computer Science.

[20]  Stefan Savage,et al.  Understanding Availability , 2003, IPTPS.

[21]  Abhinandan Das,et al.  SWIM: scalable weakly-consistent infection-style process group membership protocol , 2002, Proceedings International Conference on Dependable Systems and Networks.

[22]  David E. Culler,et al.  The ganglia distributed monitoring system: design, implementation, and experience , 2004, Parallel Comput..

[23]  D. B. Davis,et al.  Sun Microsystems Inc. , 1993 .

[24]  Ian T. Foster,et al.  On Death, Taxes, and the Convergence of Peer-to-Peer and Grid Computing , 2003, IPTPS.

[25]  Richard Wolski,et al.  The network weather service: a distributed resource performance forecasting service for metacomputing , 1999, Future Gener. Comput. Syst..

[26]  Andrew A. Chien,et al.  Entropia: architecture and performance of an enterprise desktop grid system , 2003, J. Parallel Distributed Comput..

[27]  R. Wolski,et al.  Predicting the CPU availability of time‐shared Unix systems on the computational grid , 1999, Proceedings. The Eighth International Symposium on High Performance Distributed Computing (Cat. No.99TH8469).

[28]  Wolfgang Blochinger,et al.  An Integrated Approach for Managing Peer-to-Peer Desktop Grid Systems , 2007, Seventh IEEE International Symposium on Cluster Computing and the Grid (CCGrid '07).

[29]  Samuel Madden,et al.  TAG: a Tiny Aggregation Tree for ad-hoc sensor networks , 2002, OSDI 2002.

[30]  Vaidy S. Sunderam,et al.  Towards Self-Organizing Distributed Computing Frameworks: The H2O Approach , 2003, Parallel Process. Lett..

[31]  Jeffrey D. Case,et al.  Simple Network Management Protocol (SNMP) , 1989, RFC.

[32]  David E. Culler,et al.  Proceedings of the 5th Symposium on Operating Systems Design and Implementation , 2022 .

[33]  Wolfgang Blochinger,et al.  Aspect-Oriented Parallel Discrete Optimization on the Cohesion Desktop Grid Platform , 2006, Sixth IEEE International Symposium on Cluster Computing and the Grid (CCGRID'06).

[34]  William E. Weihl,et al.  Lottery scheduling: flexible proportional-share resource management , 1994, OSDI '94.

[35]  Kenneth P. Birman The surprising power of epidemic communication , 2003 .