Reusable Meta-Models for Crowdsourcing Driven Elastic Systems (Invited Paper)

Elastic systems utilize both human and machine working units to accomplish tasks that are eligible for crowdsourcing. The quality in the results of work completed by either type of computing unit is tantamount on the characteristics they bear. In this paper we draw parallels from our previous work into looking at the suitability of working units in completing viable tasks in crowdsourcing. We seek to understand characteristics for modeling tasks and workers within these types of systems. Based on our experiments and lessons learned in related literature, we propose a dynamic worker-task information meta-model with a corresponding operational workflow model that can be used in a variety of problem domains involving crowdsourced tasks to provide support in making this decision.

[1]  M. Brian Blake,et al.  Crowdsourcing, Mixed Elastic Systems and Human-Enhanced Computing–A Survey , 2018, IEEE Transactions on Services Computing.

[2]  Kwong-Sak Leung,et al.  A Survey of Crowdsourcing Systems , 2011, 2011 IEEE Third Int'l Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third Int'l Conference on Social Computing.

[3]  Maria E. Niessen,et al.  NoiseTube: Measuring and mapping noise pollution with mobile phones , 2009, ITEE.

[4]  Martin Schader,et al.  Workers' Task Choice in Crowdsourcing and Human Computation Markets , 2012, ICIS.

[5]  Travis Desell,et al.  On the Effectiveness of Crowd Sourcing Avian Nesting Video Analysis at Wildlife@Home , 2015, ICCS.

[6]  Lydia B. Chilton,et al.  The labor economics of paid crowdsourcing , 2010, EC '10.

[7]  Schahram Dustdar,et al.  Virtualizing Software and Human for Elastic Hybrid Services , 2014, Advanced Web Services.

[8]  M. Brian Blake,et al.  Self-Generating a Labor Force for Crowdsourcing: Is Worker Confidence a Predictor of Quality? , 2015, 2015 Third IEEE Workshop on Hot Topics in Web Systems and Technologies (HotWeb).

[9]  Frank Kleemann,et al.  Un(der)paid innovators: the commercial utilization of consumer work through crowdsourcing , 2008 .

[10]  Jaime G. Carbonell,et al.  Active Learning and Crowd-Sourcing for Machine Translation , 2010, LREC.

[11]  Tim Kraska,et al.  CrowdDB: answering queries with crowdsourcing , 2011, SIGMOD '11.

[12]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[13]  Iman Saleh,et al.  Social-Network-Sourced Big Data Analytics , 2013, IEEE Internet Computing.

[14]  Iman Saleh,et al.  Combining human and machine computing elements for analysis via crowdsourcing , 2014, 10th IEEE International Conference on Collaborative Computing: Networking, Applications and Worksharing.

[15]  Alon Y. Halevy,et al.  Crowdsourcing systems on the World-Wide Web , 2011, Commun. ACM.

[16]  Deepak Ganesan,et al.  mCrowd: a platform for mobile crowdsourcing , 2009, SenSys '09.

[17]  Michael F. Goodchild,et al.  Please Scroll down for Article International Journal of Digital Earth Crowdsourcing Geographic Information for Disaster Response: a Research Frontier Crowdsourcing Geographic Information for Disaster Response: a Research Frontier , 2022 .