Serendipity: A Distributed Computing Platform for Disruption Tolerant Networks

The opportunistic or disruption tolerant networking (DTN) paradigm shows up in a variety of settings, from military to disasters to the developing world to deep space; anywhere that fixed infrastructure is either unavailable or expensiv e. Simple messaging applications have substantial value for c ommunication and coordination. We posit that these settings can also leverage applications that are computationally co mplex and will benefit from distributed computing to take full advantage of nearby computational resources. Computing over these networks is not trivial, however, since network disconnections are common and persist over many time scales . In this paper we present the design and implementation of Serendipity, a general purpose distributed computing plat form designed to run over a DTN. We have designed a simple but powerful job structure that is suitable for such an un derlying network. As opposed to traditional distributed co mputing platforms in data centers and clusters, where a central master is used to allocate tasks and monitor the working nodes, Serendipity relies on the collaboration among DTN nodes on these functionalities. Smart task allocation algo rithms are designed to disseminate tasks among mobile devices by accounting for the special properties of DTNs. The extensive evaluation of our system on Emulab demonstrates that Serendipity efficiently speeds up various kinds of distributed computing jobs by a factor of 2.3 to 10.1 in a diverse set of DTN environments.

[1]  Srinivasan Keshav,et al.  Very low-cost internet access using KioskNet , 2007, CCRV.

[2]  Miron Livny,et al.  Condor-a hunter of idle workstations , 1988, [1988] Proceedings. The 8th International Conference on Distributed.

[3]  Arun Venkataramani,et al.  Web search from a bus , 2007, CHANTS '07.

[5]  Peter J. Denning Hastily formed networks , 2006, CACM.

[6]  Alex Pentland,et al.  DakNet: rethinking connectivity in developing nations , 2004, Computer.

[7]  Stephen Farrell,et al.  DTN: an architectural retrospective , 2008, IEEE Journal on Selected Areas in Communications.

[8]  David P. Anderson,et al.  SETI@home: an experiment in public-resource computing , 2002, CACM.

[9]  R. Wade,et al.  Mitigating naval network instabilities with disruption toler , 2008, MILCOM 2008 - 2008 IEEE Military Communications Conference.

[10]  Vinton G. Cerf,et al.  Delay-tolerant networking: an approach to interplanetary Internet , 2003, IEEE Commun. Mag..

[11]  Srinivasan Keshav,et al.  Low-cost communication for rural internet kiosks using mechanical backhaul , 2006, MobiCom '06.

[12]  R. Manmatha,et al.  Mobile distributed information retrieval for highly-partitioned networks , 2003, 11th IEEE International Conference on Network Protocols, 2003. Proceedings..

[13]  Mike Hibler,et al.  An integrated experimental environment for distributed systems and networks , 2002, OPSR.

[14]  Brian Gallagher,et al.  MaxProp: Routing for Vehicle-Based Disruption-Tolerant Networks , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[15]  Kyunghan Lee,et al.  On the Levy-Walk Nature of Human Mobility , 2008, IEEE INFOCOM 2008 - The 27th Conference on Computer Communications.

[16]  Rabin K. Patra,et al.  Routing in a delay tolerant network , 2004, SIGCOMM '04.

[17]  Arun Venkataramani,et al.  Interactive wifi connectivity for moving vehicles , 2008, SIGCOMM '08.

[18]  Chris Jackson,et al.  Experience with Delay-Tolerant Networking from Orbit , 2008 .

[19]  Pan Hui,et al.  Haggle: A networking architecture designed around mobile users , 2006 .

[20]  Hsiao-Wuen Hon,et al.  An overview of the SPHINX speech recognition system , 1990, IEEE Trans. Acoust. Speech Signal Process..

[21]  Narayanan Sadagopan,et al.  the IMPORTANT framework for analyzing the Impa t ofMobility on Performan e of RouTing proto ols for Adho , 2003 .

[22]  Scott C. Burleigh,et al.  Bundle Protocol Specification , 2007, RFC.

[23]  Eugene Marinelli,et al.  Hyrax: Cloud Computing on Mobile Devices using MapReduce , 2009 .

[24]  Yuan Yu,et al.  Dryad: distributed data-parallel programs from sequential building blocks , 2007, EuroSys '07.

[25]  Michael J. Demmer,et al.  DTLSR: delay tolerant routing for developing regions , 2007, NSDR '07.

[26]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[27]  Sanjay P. Ahuja,et al.  A Survey on Wireless Grid Computing , 2006, The Journal of Supercomputing.

[28]  Zhensheng Zhang,et al.  Routing in intermittently connected mobile ad hoc networks and delay tolerant networks: overview and challenges , 2006, IEEE Communications Surveys & Tutorials.

[29]  David P. Anderson,et al.  BOINC: a system for public-resource computing and storage , 2004, Fifth IEEE/ACM International Workshop on Grid Computing.

[30]  Amit Kumar Saha,et al.  Modeling mobility for vehicular ad-hoc networks , 2004, VANET '04.

[31]  S. Parikh,et al.  Disruption tolerant networking for Marine Corps CONDOR , 2005, MILCOM 2005 - 2005 IEEE Military Communications Conference.

[32]  Vijay S. Pande,et al.  Folding@home: Lessons from eight years of volunteer distributed computing , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[33]  Francine Berman,et al.  Heuristics for scheduling parameter sweep applications in grid environments , 2000, Proceedings 9th Heterogeneous Computing Workshop (HCW 2000) (Cat. No.PR00556).

[34]  Ahmed Helmy,et al.  Modeling Time-Variant User Mobility in Wireless Mobile Networks , 2007, IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications.

[35]  Fernando Silveira,et al.  A disruption-tolerant architecture for secure and efficient disaster response communications , 2010, ISCRAM.

[36]  Miron Livny,et al.  Distributed computing in practice: the Condor experience: Research Articles , 2005 .

[37]  P. Marshall DARPA progress towards affordable, dense, and content focused tactical edge networks , 2008, MILCOM 2008 - 2008 IEEE Military Communications Conference.