Fueling AI with public displays?: a feasibility study of collecting biometrically tagged consensual data on a university campus

Interactive public displays have matured into highly capable two-way interfaces. They can be used for efficiently delivering information to people as well as for collecting insights from their users. While displays have been used for harvesting opinions and other content from users, surprisingly little work has looked into exploiting such screens for the consensual collection of tagged data that might be useful beyond one application. We present a field study where we collected biometrically tagged data using public kiosk-sized interactive screens. During 61 days of deployment time, we collected 199 selfie videos, cost-efficiently and with consent to leverage the videos in any non-profit research. 78 of the videos also had metadata attached to them. Overall, our studies indicate that people are willing to donate even highly sensitive data about themselves in public but that, at the same time, the participants had specific ethical and privacy concerns over the future of their data. Our study paves the way forward toward a future where volunteers can ethically help advance innovations in computer vision research across a variety of exciting application domains, such as health monitoring and care.

[1]  M. Swan Health 2050: The Realization of Personalized Medicine through Crowdsourcing, the Quantified Self, and the Participatory Biocitizen , 2012, Journal of personalized medicine.

[2]  Florian Alt,et al.  EyeVote in the wild: do users bother correcting system errors on public displays? , 2016, MUM.

[3]  Jorge Gonçalves,et al.  Application discoverability on multipurpose public displays: popularity comes at a price , 2013, PerDis.

[4]  A. Acquisti,et al.  Beyond the Turk: Alternative Platforms for Crowdsourcing Behavioral Research , 2016 .

[5]  Jorge Gonçalves,et al.  Monetary Assessment of Battery Life on Smartphones , 2016, CHI.

[6]  Jorge Gonçalves,et al.  Situated crowdsourcing using a market model , 2014, UIST.

[7]  J. Domingo-Ferrer,et al.  Beyond Informed Consent—Investigating Ethical Justifications for Disclosing, Donating or Sharing Personal Data in Research , 2017 .

[8]  Jorge Gonçalves,et al.  CrowdPickUp , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[9]  R. McAfee,et al.  Auctions and Bidding , 1986 .

[10]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[11]  Judy Kay,et al.  Who cares about the Content? An Analysis of Playful Behaviour at a Public Display , 2014, PerDis.

[12]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Kathy Ryall,et al.  UbiTable: Impromptu Face-to-Face Collaboration on Horizontal Interactive Surfaces , 2003, UbiComp.

[14]  Alireza Sahami Shirazi,et al.  Digifieds: insights into deploying digital public notice areas in the wild , 2011, MUM.

[15]  Simo Hosio,et al.  Experizone: Integrating Situated Scientific Experimentation with Teaching of the Scientific Method , 2019, CHI Extended Abstracts.

[16]  Jukka Riekki,et al.  Crowdsourcing Public Opinion Using Urban Pervasive Technologies: Lessons From Real‐Life Experiments in Oulu , 2015 .

[17]  O. O’neill,et al.  Some limits of informed consent , 2003, Journal of medical ethics.

[18]  Nemanja Memarovic,et al.  Longitudinal, cross-site and "in the Wild": a study of public displays user communities' situated snapshots , 2016, MAB.

[19]  Florian Alt,et al.  Looking glass: a field study on noticing interactivity of a shop window , 2012, CHI.

[20]  Jorge Gonçalves,et al.  Eliciting situated feedback: A comparison of paper, web forms and public displays , 2014, Displays.

[21]  Jorge Gonçalves,et al.  Multipurpose Public Displays: How Shortcut Menus Affect Usage , 2013, IEEE Computer Graphics and Applications.

[22]  Casey Fiesler,et al.  "We Are the Product": Public Reactions to Online Data Sharing and Privacy Controversies in the Media , 2018, CHI.

[23]  Jorge Gonçalves,et al.  Facilitating Collocated Crowdsourcing on Situated Displays , 2018, Hum. Comput. Interact..

[24]  Björn Hartmann,et al.  CommunitySourcing: engaging local crowds to perform expert work via physical kiosks , 2012, CHI.

[25]  Alessandro Acquisti,et al.  Imagined Communities: Awareness, Information Sharing, and Privacy on the Facebook , 2006, Privacy Enhancing Technologies.

[26]  Christian Weichel,et al.  Overcoming interaction blindness through curiosity objects , 2013, CHI Extended Abstracts.

[27]  Jorge Gonçalves,et al.  Crowdsourcing on the spot: altruistic use of public displays, feasibility, performance, and behaviours , 2013, UbiComp.

[28]  Albrecht Schmidt,et al.  Designing Shared Public Display Networks - Implications from Today's Paper-Based Notice Areas , 2011, Pervasive.

[29]  Albrecht Schmidt,et al.  Requirements and design space for interactive public displays , 2010, ACM Multimedia.

[30]  Alison R. T. Brenner,et al.  Beyond Informed Consent: Educating the Patient , 2009, Clinical orthopaedics and related research.

[31]  Abdenour Hadid,et al.  A Survey on Computer Vision for Assistive Medical Diagnosis From Faces , 2018, IEEE Journal of Biomedical and Health Informatics.

[32]  M. Goldberg,et al.  Validity of self-reported weight and height in the French GAZEL cohort , 2000, International Journal of Obesity.

[33]  Nicu Sebe,et al.  Money walks: a human-centric study on the economics of personal mobile data , 2014, UbiComp.

[34]  Aditya G. Parameswaran,et al.  Optimizing Open-Ended Crowdsourcing: The Next Frontier in Crowdsourced Data Management , 2016, IEEE Data Eng. Bull..

[35]  Christiane Lehrer,et al.  Digital Nudging: Altering User Behavior in Digital Environments , 2017, Wirtschaftsinformatik.