Characterising volunteers' task execution patterns across projects on multi-project citizen science platforms

Citizen science projects engage people in activities that are part of a scientific research effort. On multi-project citizen science platforms, scientists can create projects consisting of tasks. Volunteers, in turn, participate in executing the project's tasks. Such type of platforms seeks to connect volunteers and scientists' projects, adding value to both. However, little is known about volunteer's cross-project engagement patterns and the benefits of such patterns for scientists and volunteers. This work proposes a Goal, Question, and Metric (GQM) approach to analyse volunteers' cross-project task execution patterns and employs the Semiotic Inspection Method (SIM) to analyse the communicability of the platform's cross-project features. In doing so, it investigates what are the features of platforms to foster volunteers' cross-project engagement, to what extent multi-project platforms facilitate the attraction of volunteers to perform tasks in new projects, and to what extent multi-project participation increases engagement on the platforms. Results from analyses on real platforms show that volunteers tend to explore multiple projects, but they perform tasks regularly in just a few of them; few projects attract much attention from volunteers; volunteers recruited from other projects on the platform tend to get more engaged than those recruited outside the platform. System inspection shows that platforms still lack personalised and explainable recommendations of projects and tasks.

[1]  David De Roure,et al.  Zooniverse: observing the world's largest citizen science platform , 2014, WWW.

[2]  H. D. Rombach,et al.  The Goal Question Metric Approach , 1994 .

[3]  Egon Berghout,et al.  The Goal/Question/Metric method: a practical guide for quality improvement of software development , 1999 .

[4]  M. Edelstein Citizen Science: A Study of People, Expertise and Sustainable Development , 1998 .

[5]  Jennifer Preece,et al.  Citizen Science: New Research Challenges for Human–Computer Interaction , 2016, Int. J. Hum. Comput. Interact..

[6]  S. West,et al.  Recruiting and Retaining Participants in Citizen Science: What Can Be Learned from the Volunteering Literature? , 2016 .

[7]  François Bry,et al.  Human computation , 2018, it Inf. Technol..

[8]  Clarisse Sieckenius de Souza,et al.  The Semiotic Engineering of Human-Computer Interaction , 2005 .

[9]  D. Roy,et al.  Patterns of contribution to citizen science biodiversity projects increase understanding of volunteers’ recording behaviour , 2016, Scientific Reports.

[10]  Ann Blandford,et al.  Designing for dabblers and deterring drop-outs in citizen science , 2014, CHI.

[11]  Jessica L. Cappadonna,et al.  Citizen Science Terminology Matters: Exploring Key Terms , 2017, Citizen Science: Theory and Practice.

[12]  Lesandro Ponciano,et al.  Finding Volunteers' Engagement Profiles in Human Computation for Citizen Science Projects , 2014, Hum. Comput..

[13]  C. Lintott,et al.  Galaxy Zoo: Exploring the Motivations of Citizen Science Volunteers. , 2009, 0909.2925.

[14]  G TomsElaine,et al.  What is user engagement? A conceptual framework for defining user engagement with technology , 2008 .

[15]  Edna Dias Canedo,et al.  Interaction Design Process Oriented by Metrics , 2018, HCI.

[16]  Ricardo Matsumura de Araújo,et al.  99designs: An Analysis of Creative Competition in Crowdsourced Design , 2013, HCOMP.

[17]  Lesandro Ponciano,et al.  Volunteers' Engagement in Human Computation for Astronomy Projects , 2014, Computing in Science & Engineering.

[18]  Panagiotis G. Ipeirotis,et al.  Demographics and Dynamics of Mechanical Turk Workers , 2018, WSDM.

[19]  Lesandro Ponciano,et al.  Agreement-based credibility assessment and task replication in human computation systems , 2018, Future Gener. Comput. Syst..

[20]  M. Hardaway,et al.  Volunteering. , 2020, The Veterinary record.

[21]  Chris Lintott,et al.  Human Computation in Citizen Science , 2013, Handbook of Human Computation.

[22]  Pietro Michelucci,et al.  The power of crowds , 2016, Science.

[23]  Clarisse Sieckenius de Souza,et al.  The semiotic inspection method , 2006, IHC '06.

[24]  Francisco Vilar Brasileiro,et al.  An Analysis of the Use of Qualifications on the Amazon Mechanical Turk Online Labor Market , 2017, Computer Supported Cooperative Work (CSCW).

[25]  Manuel Blum,et al.  reCAPTCHA: Human-Based Character Recognition via Web Security Measures , 2008, Science.

[26]  M. Haklay Citizen Science and Volunteered Geographic Information: Overview and Typology of Participation , 2013 .

[27]  Elaine Toms,et al.  What is user engagement? A conceptual framework for defining user engagement with technology , 2008, J. Assoc. Inf. Sci. Technol..

[28]  Lesandro Ponciano,et al.  Considering human aspects on strategies for designing and managing distributed human computation , 2014, Journal of Internet Services and Applications.

[29]  Arthur A. Stukas,et al.  PERSONALITY PROCESSES AND INDIVIDUAL DIFFERENCES Understanding and Assessing the Motivations of Volunteers: A Functional Approach , 2004 .

[30]  Rini van Solingen Agile GQM: Why Goal/Question/Metric is more Relevant than Ever and Why It Helps Solving the Agility Challenges of Today's Organizations , 2014, IWSM/Mensura.

[31]  John Darlington,et al.  Design Guidelines for the User-Centred Collaborative Citizen Science Platforms , 2016, Hum. Comput..

[32]  Kevin Crowston,et al.  From Conservation to Crowdsourcing: A Typology of Citizen Science , 2011, 2011 44th Hawaii International Conference on System Sciences.

[33]  Kevin Crowston,et al.  Goals and Tasks: Two Typologies of Citizen Science Projects , 2012, 2012 45th Hawaii International Conference on System Sciences.

[34]  Marti A. Hearst,et al.  Comparing the use of social networking and traditional media channels for promoting citizen science , 2013, CSCW.

[35]  Jennifer Preece,et al.  Dynamic changes in motivation in collaborative citizen-science projects , 2012, CSCW.

[36]  Anita Greenhill,et al.  Defining and Measuring Success in Online Citizen Science: A Case Study of Zooniverse Projects , 2015, Computing in Science & Engineering.