SDIMS: A Scalable Distributed Information Management System

January 23, 2007 1 SDIMS: A Scalable Distributed Information Management System Courtesy of Praveen Yalagandula (UT Austin) Varun Singhal January 23, 2007 2 Goal A Distributed Operating System Backbone Information collection and management Core functionality of large distributed systems Monitor, query and react to changes in the system Examples: Benefits Ease development of new services Facilitate deployment Avoid repetition of same task by different services Optimize system performance System administration and management Service placement and location Sensor monitoring and control Distributed Denial-of-Service attack detection File location service Multicast tree construction Naming and request routing ............

[1]  Deborah Estrin,et al.  Directed diffusion: a scalable and robust communication paradigm for sensor networks , 2000, MobiCom '00.

[2]  Peter Druschel,et al.  Exploiting network proximity in peer-to-peer overlay networks , 2002 .

[3]  Kenneth P. Birman,et al.  The Surprising Power of Epidemic Communication , 2003, Future Directions in Distributed Computing.

[4]  Robbert van Renesse,et al.  Astrolabe: A robust and scalable technology for distributed system monitoring, management, and data mining , 2003, TOCS.

[5]  Michael Dahlin,et al.  Design considerations for distributed caching on the Internet , 1999, Proceedings. 19th IEEE International Conference on Distributed Computing Systems (Cat. No.99CB37003).

[6]  Indranil Gupta,et al.  Kelips: Building an Efficient and Stable P2P DHT through Increased Memory and Background Overhead , 2003, IPTPS.

[7]  Moni Naor,et al.  Viceroy: a scalable and dynamic emulation of the butterfly , 2002, PODC '02.

[8]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[9]  Wei Hong,et al.  Proceedings of the 5th Symposium on Operating Systems Design and Implementation Tag: a Tiny Aggregation Service for Ad-hoc Sensor Networks , 2022 .

[10]  Burton H. Bloom,et al.  Space/time trade-offs in hash coding with allowable errors , 1970, CACM.

[11]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[12]  David E. Culler,et al.  The ganglia distributed monitoring system: design, implementation, and experience , 2004, Parallel Comput..

[13]  Ben Y. Zhao,et al.  An Infrastructure for Fault-tolerant Wide-area Location and Routing , 2001 .

[14]  James Aspnes,et al.  Skip graphs , 2003, SODA '03.

[15]  Krishna P. Gummadi,et al.  The impact of DHT routing geometry on resilience and proximity , 2003, SIGCOMM '03.

[16]  David R. Karger,et al.  Koorde: A Simple Degree-Optimal Distributed Hash Table , 2003, IPTPS.

[17]  Miguel Castro,et al.  Scribe: a large-scale and decentralized application-level multicast infrastructure , 2002, IEEE J. Sel. Areas Commun..

[18]  Marvin Theimer,et al.  Flexible update propagation for weakly consistent replication , 1997, SOSP.

[19]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.

[20]  Peter Druschel,et al.  Pastry: Scalable, distributed object location and routing for large-scale peer-to- , 2001 .

[21]  Xiaozhou Li,et al.  On name resolution in peer-to-peer networks , 2002, POMC '02.

[22]  D. Malkhi,et al.  Dynamic lookup networks , 2003 .

[23]  Mark Handley,et al.  Application-Level Multicast Using Content-Addressable Networks , 2001, Networked Group Communication.

[24]  Gurmeet Singh Manku,et al.  Symphony: Distributed Hashing in a Small World , 2003, USENIX Symposium on Internet Technologies and Systems.

[25]  Amin Vahdat,et al.  SHARP: an architecture for secure resource peering , 2003, SOSP '03.

[26]  Timothy Roscoe,et al.  InfoSpect: using a logic language for system health monitoring in distributed systems , 2002, EW 10.

[27]  David R. Karger,et al.  Wide-area cooperative storage with CFS , 2001, SOSP.

[28]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[29]  Ben Y. Zhao,et al.  OceanStore: an architecture for global-scale persistent storage , 2000, SIGP.

[30]  Rajkumar Buyya,et al.  PARMON: a portable and scalable monitoring system for clusters , 2000, Softw. Pract. Exp..

[31]  Ben Y. Zhao,et al.  Towards a Common API for Structured Peer-to-Peer Overlays , 2003, IPTPS.

[32]  Michael B. Jones,et al.  Efficient Recovery from Organizational Disconnects in SkipNet , 2003, IPTPS.

[33]  Rajmohan Rajaraman,et al.  Accessing Nearby Copies of Replicated Objects in a Distributed Environment , 1997, SPAA '97.

[34]  Miguel Castro,et al.  SplitStream: high-bandwidth multicast in cooperative environments , 2003, SOSP '03.

[35]  Richard Wolski,et al.  The network weather service: a distributed resource performance forecasting service for metacomputing , 1999, Future Gener. Comput. Syst..

[36]  Arun Iyengar,et al.  A Scalable and Highly Available System for Serving Dynamic Data at Frequently Accessed Web Sites , 1998, Proceedings of the IEEE/ACM SC98 Conference.

[37]  Michael B. Jones,et al.  SkipNet: A Scalable Overlay Network with Practical Locality Properties , 2003, USENIX Symposium on Internet Technologies and Systems.

[38]  G. Weikum.  Querying the Internet with PIER , 2005 .

[39]  Larry L. Peterson,et al.  Sophia: an Information Plane for networked systems , 2004, Comput. Commun. Rev..

[40]  Antony I. T. Rowstron,et al.  Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility , 2001, SOSP.

[41]  Dan Suciu,et al.  What Can Peer-to-Peer Do for Databases, and Vice Versa? , 2001 .

[42]  Robert Tappan Morris,et al.  Serving DNS Using a Peer-to-Peer Lookup Service , 2002, IPTPS.

[43]  Ben Y. Zhao,et al.  Bayeux: an architecture for scalable and fault-tolerant wide-area data dissemination , 2001, NOSSDAV '01.