Leveraging Data Intensive Applications on a Pervasive Computing Platform: The Case of MapReduce

Abstract Pervasive grids represent an important step towards the establishment of mobiquitous systems, but at the same time these environments are especially challenging when considering data distribution and data intensive processing. In this work we present CloudFIT, a middleware designed to support the volatility of pervasive environments, and discuss the main challenges related to efficiently deploy MapReduce applications on pervasive environments.

[1]  Gilles Fedak,et al.  Optimizing Data Distribution in Desktop Grid Platforms , 2008, Parallel Process. Lett..

[2]  Peter Druschel,et al.  Pastry: Scalable, distributed object location and routing for large-scale peer-to- , 2001 .

[3]  Sergio Nesmachnow,et al.  Mapreduce Challenges on Pervasive Grids , 2014, J. Comput. Sci..

[4]  Daniel Diaz,et al.  PER-MARE: Adaptive Deployment of MapReduce over Pervasive Grids , 2013, 2013 Eighth International Conference on P2P, Parallel, Grid, Cloud and Internet Computing.

[5]  Jean-Marc Pierson,et al.  Pervasive Grids Challenges and Opportunities , 2008 .

[6]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[7]  T. N. Vijaykumar,et al.  Tarazu: optimizing MapReduce on heterogeneous clusters , 2012, ASPLOS XVII.

[8]  Randy H. Katz,et al.  Improving MapReduce Performance in Heterogeneous Environments , 2008, OSDI.

[9]  Eugenio Cesario,et al.  Distributed Data Mining using a Public Resource Computing Framework , 2009, CoreGRID@Euro-Par.

[10]  Vijay S. Pande,et al.  Folding@home: Lessons from eight years of volunteer distributed computing , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[11]  Stephen L. Scott,et al.  FreeLoader: Scavenging Desktop Storage Resources for Scientific Data , 2005, ACM/IEEE SC 2005 Conference (SC'05).

[12]  Ian Taylor,et al.  A peer-to-peer architecture for data-intensive cycle sharing , 2011, NDM '11.

[13]  David P. Anderson,et al.  SETI@home: an experiment in public-resource computing , 2002, CACM.

[14]  Wu-chun Feng,et al.  MOON: MapReduce On Opportunistic eNvironments , 2010, HPDC '10.

[15]  Giuseppe De Pietro,et al.  MiPeG: A middleware infrastructure for pervasive grids , 2008, Future Gener. Comput. Syst..

[16]  Domenico Talia,et al.  A scalable super-peer approach for public scientific computation , 2009, Future Gener. Comput. Syst..

[17]  Jack Dongarra,et al.  Handbook of Research on Scalable Computing Technologies , 2009 .

[18]  Domenico Talia,et al.  A Peer-to-Peer Framework for Supporting MapReduce Applications in Dynamic Cloud Environments , 2010, Cloud Computing.

[19]  Antony I. T. Rowstron,et al.  Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility , 2001, SOSP.

[20]  Luiz Angelo Steffenel.  First Steps on the Development of a P2P Middleware for Map-Reduce , 2013 .

[21]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[22]  Quan Chen,et al.  SAMR: A Self-adaptive MapReduce Scheduling Algorithm in Heterogeneous Environment , 2010, 2010 10th IEEE International Conference on Computer and Information Technology.