Improving Accuracy in Mobile Human Contributions: An Overview

The collection of human contributions through mobile devices is increasingly common across a range of methodologies. However, possible quality issues of these contributions are often overlooked. As the quality of human data has a direct impact on study reliability, more should be done to improve the accuracy of these contributions. We identify and categorise solutions aimed at increasing the accuracy of contributions prior, during, and following data collection. Our categorisation assists in the positioning of future work in this area and fosters the usage of cross-methodological practises.

[1]  Aniket Kittur,et al.  Crowdsourcing user studies with Mechanical Turk , 2008, CHI.

[2]  Jorge Gonçalves,et al.  Gamification of Mobile Experience Sampling Improves Data Quality and Quantity , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[3]  R. Kessler,et al.  Measuring daily events and experiences: decisions for the researcher. , 1991, Journal of personality.

[4]  Carolyn E. Schwartz,et al.  Exploring response shift in longitudinal data , 2004 .

[5]  Matthias Budde,et al.  Sensified Gaming: Design Patterns and Game Design Elements for Gameful Environmental Sensing , 2016, ACE.

[6]  Mirco Musolesi,et al.  Ask, but don't interrupt: the case for interruptibility-aware mobile experience sampling , 2015, UbiComp/ISWC Adjunct.

[7]  Matthias Budde,et al.  Participatory Sensing or Participatory Nonsense? , 2017 .

[8]  Jorge Gonçalves,et al.  Facilitating Collocated Crowdsourcing on Situated Displays , 2018, Hum. Comput. Interact..

[9]  Matthias Budde,et al.  Lessons from failures in designing and conducting experimental studies: a brief anecdotal tutorial , 2017, UbiComp/ISWC Adjunct.

[10]  Jorge Gonçalves,et al.  Mobile Decision Support and Data Provisioning for Low Back Pain , 2018, Computer.

[11]  Reza Curtmola,et al.  Crowdsensing in the Wild with Aliens and Micropayments , 2016, IEEE Pervasive Computing.

[12]  Sunny Consolvo,et al.  Using the Experience Sampling Method to Evaluate Ubicomp Applications , 2003, IEEE Pervasive Comput..

[13]  Cecilia Mascolo,et al.  Contextual dissonance: design bias in sensor-based experience sampling methods , 2013, UbiComp.

[14]  Gargi Dasgupta,et al.  Citicafe: An Interactive Interface for Citizen Engagement , 2018, IUI.

[15]  Michaela Riediger,et al.  Experience Sampling , 2009, SAGE Research Methods Foundations.

[16]  Jie Wu,et al.  QoI-Aware Multitask-Oriented Dynamic Participant Selection With Budget Constraints , 2014, IEEE Transactions on Vehicular Technology.

[17]  Niels van Berkel,et al.  The Experience Sampling Method on Mobile Devices , 2017, ACM Comput. Surv..

[18]  Scott R. Klemmer,et al.  Shepherding the crowd yields better work , 2012, CSCW.

[19]  Anne M. Land-Zandstra,et al.  Citizen science on a smartphone: Participants’ motivations and learning , 2016, Public understanding of science.

[20]  Matthias Budde,et al.  Participatory Sensing or Participatory Nonsense?: Mitigating the Effect of Human Error on Data Quality in Citizen Science , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..