Resource Adaptive Distributed Information Sharing

We have designed, implemented and evaluated a resource adaptive distributed information sharing system where automatic adjustments are made internally in our information sharing system in order to cope with varying resource consumption. CPU load is monitored and a light-weight trigger mechanism is used to avoid overload situations on a per-machine basis. Additional improvements are obtained by calculating what we call a utility score to better determine how the data structures in the system should be arranged. Our results show that resource adaptation is an efficient way of improving query throughput, and that it is most effective when the number of stored data items in the system is large or many queries are performed concurrently. By applying resource adaptation, we are able to significantly improve the performance of our information sharing system.

[1]  Domenico Ferrari,et al.  An Empirical Investigation of Load Indices for Load Balancing Applications , 1987, Performance.

[2]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[3]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[4]  Robbert van Renesse,et al.  Astrolabe: A robust and scalable technology for distributed system monitoring, management, and data mining , 2003, TOCS.

[5]  Srinivasan Seshan,et al.  Mercury: supporting scalable multi-attribute range queries , 2004, SIGCOMM 2004.

[6]  Praveen Yalagandula,et al.  A scalable distributed information management system , 2004, SIGCOMM 2004.

[7]  Srinivasan Seshan,et al.  Mercury: supporting scalable multi-attribute range queries , 2004, SIGCOMM '04.

[8]  John Kubiatowicz,et al.  Handling churn in a DHT , 2004 .

[9]  Stephen Tallman,et al.  THE EFFECTS OF KNOWLEDGE STRATEGY AND INTERNATIONAL DIVERSITY ON MNES' PERFORMANCE AFTER THE SHOCK OF SEP 11 TH ATTACKS. , 2006 .

[10]  Ramesh Govindan,et al.  MIND: A Distributed Multi-Dimensional Indexing System for Network Diagnosis , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[11]  Scott Shenker,et al.  A data-oriented (and beyond) network architecture , 2007, SIGCOMM '07.

[12]  Klara Nahrstedt,et al.  Self-Configuring Information Management for Large-Scale Service Overlays , 2007, IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications.

[13]  Jussi Kangasharju,et al.  Optimizing File Availability in Peer-to-Peer Content Distribution , 2007, IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications.

[14]  Ming Zhong,et al.  Replication degree customization for high availability , 2008, Eurosys '08.

[15]  Pekka Nikander,et al.  LIPSIN: line speed publish/subscribe inter-networking , 2009, SIGCOMM '09.

[16]  Cheng-Zhong Xu,et al.  Elastic Routing Table with Provable Performance for Congestion Control in DHT Networks , 2010, IEEE Trans. Parallel Distributed Syst..