Distributed computing by leveraging and rewarding idling user resources from P2P networks

Abstract Currently, many emerging computer science applications call for collaborative solutions to complex projects that require huge amounts of computing resources to be completed, e.g., physical science simulation, big data analysis. Many approaches have been proposed for high performance computing designing a task partitioning strategy able to assign pieces of execution to the appropriate workers in order to parallelize task execution. In this paper, we describe the Coremuniti T M system, our peer to peer solution for solving complex works by using the idling computational resources of users connected to our network. More in detail, we designed a framework that allows users to share their CPU and memory in a secure and efficient way. By doing this, users help each other by asking the network computational resources when they face high computing demanding tasks. In this respect, as users provide their computational power without providing specific human skill, our approach can be considered as a hybrid crowdsourcing. Differently from many proposals available for volunteer computing, users providing their resources are rewarded with tangible credits, i.e., they can redeem their credits by asking for computational power to solve their own task and/or by exchanging them for money. We conducted a comprehensive experimental assessment in an interesting scenario as 3D rendering, which allowed us to validate the scalability and effectiveness of our solution and its profitability for end-users. As we do not require to power additional resources for solving tasks (we better exploit unused resources already powered instead), we hypothesize a remarkable side effect at steady state: energy consumption reduction compared with traditional server farms or cloud based executions.

[1]  David R. Karger,et al.  Counting with the Crowd , 2012, Proc. VLDB Endow..

[2]  Min Yang,et al.  An Efficient Hybrid Peer-to-Peer System for Distributed Data Sharing , 2010, IEEE Trans. Computers.

[3]  J. Wenny Rahayu,et al.  Mobile cloud computing: A survey , 2013, Future Gener. Comput. Syst..

[4]  Purnamrita Sarkar,et al.  Scaling Up Crowd-Sourcing to Very Large Datasets: A Case for Active Learning , 2014, Proc. VLDB Endow..

[5]  R. Burnham,et al.  Threshold and suprathreshold perceptual color differences. , 1973, Journal of the Optical Society of America.

[6]  Hector Garcia-Molina,et al.  Comparing Hybrid Peer-to-Peer Systems , 2001, VLDB.

[7]  Sartaj Sahni,et al.  Strassen's Matrix Multiplication on GPUs , 2011, 2011 IEEE 17th International Conference on Parallel and Distributed Systems.

[8]  Adam Marcus,et al.  Argonaut: Macrotask Crowdsourcing for Complex Data Processing , 2015, Proc. VLDB Endow..

[9]  Kevin Skadron,et al.  Scalable parallel programming , 2008, 2008 IEEE Hot Chips 20 Symposium (HCS).

[10]  Guoliang Li,et al.  Crowdsourced Top-k Algorithms: An Experimental Evaluation , 2016, Proc. VLDB Endow..

[11]  Alon Y. Halevy,et al.  Crowdsourcing systems on the World-Wide Web , 2011, Commun. ACM.

[12]  Jeff Howe,et al.  Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business , 2008, Human Resource Management International Digest.

[13]  Reynold Cheng,et al.  QASCA: A Quality-Aware Task Assignment System for Crowdsourcing Applications , 2015, SIGMOD Conference.

[14]  Sihem Amer-Yahia,et al.  Worker Skill Estimation in Team-Based Tasks , 2015, Proc. VLDB Endow..

[15]  Rashmi Bhatia,et al.  Grid Computing and Security Issues , 2013 .

[16]  Keke Gai,et al.  Energy-aware task assignment for mobile cyber-enabled applications in heterogeneous cloud computing , 2018, J. Parallel Distributed Comput..

[17]  Shuang Ding,et al.  Multiobjective Optimization Model for Service Node Selection Based on a Tradeoff Between Quality of Service and Resource Consumption in Mobile Crowd Sensing , 2017, IEEE Internet of Things Journal.

[18]  Keke Gai,et al.  Spoofing-Jamming Attack Strategy Using Optimal Power Distributions in Wireless Smart Grid Networks , 2017, IEEE Transactions on Smart Grid.

[19]  Sihem Amer-Yahia,et al.  Task assignment optimization in knowledge-intensive crowdsourcing , 2015, The VLDB Journal.

[20]  Lei Chen,et al.  gMission: A General Spatial Crowdsourcing Platform , 2014, Proc. VLDB Endow..

[21]  Min Chen,et al.  SA-EAST , 2017, ACM Trans. Embed. Comput. Syst..

[22]  Beng Chin Ooi,et al.  CDAS: A Crowdsourcing Data Analytics System , 2012, Proc. VLDB Endow..

[23]  Tim Kraska,et al.  CrowdER: Crowdsourcing Entity Resolution , 2012, Proc. VLDB Endow..

[24]  Keke Gai,et al.  Dynamic energy-aware cloudlet-based mobile cloud computing model for green computing , 2016, J. Netw. Comput. Appl..

[25]  Siddharth Suri,et al.  Conducting behavioral research on Amazon’s Mechanical Turk , 2010, Behavior research methods.

[26]  Ting Wu,et al.  Hear the Whole Story: Towards the Diversity of Opinion in Crowdsourcing Markets , 2015, Proc. VLDB Endow..

[27]  David R. Karger,et al.  Human-powered Sorts and Joins , 2011, Proc. VLDB Endow..

[28]  Eugene Marinelli,et al.  Hyrax: Cloud Computing on Mobile Devices using MapReduce , 2009 .

[29]  Andrew S. Glassner,et al.  An introduction to ray tracing , 1989 .

[30]  Jizhong Zhao,et al.  Reliable Diversity-Based Spatial Crowdsourcing by Moving Workers , 2014, Proc. VLDB Endow..