Revisiting Citizen Science Through the Lens of Hybrid Intelligence

Arti cial Intelligence (AI) can augment and sometimes even replace human cognition. Inspired by e orts to value human agency alongside productivity, we discuss the bene ts of solving Citizen Science (CS) tasks with Hybrid Intelligence (HI), a synergetic mixture of human and arti cial intelligence. Currently there is no clear framework or methodology on how to create such an e ective mixture. Due to the unique participant-centered set of values and the abundance of tasks drawing upon both human common sense and complex 21st century skills, we believe that the eld of CS o ers an invaluable testbed for the development of HI and human-centered AI of the 21st century, while bene ting CS as well. In order to investigate this potential, we rst relate CS to adjacent computational disciplines. Then, we demonstrate that CS projects can be grouped according to their potential for HI-enhancement by examining two key dimensions: the level of digitization and the amount of knowledge or experience required for participation. Finally, we propose a framework for types of human-AI interaction in CS based on established criteria of HI. This “HI lens” provides the CS community with an overview of several ways to utilize the combination of AI and human intelligence in their projects. It also allows the AI community to gain ideas on how developing AI in CS projects can further their own eld.

[1]  Lydia Manikonda,et al.  Complementing the Execution of AI Systems with Human Computation , 2017, AAAI Workshops.

[2]  Manuel Corpas,et al.  Lessons from Fraxinus, a crowd-sourced citizen science game in genomics , 2015, eLife.

[3]  James R. Wootton.  Getting the public involved in Quantum Error Correction , 2017, 1712.09649.

[4]  Vickie Curtis,et al.  Motivation to Participate in an Online Citizen Science Game , 2015 .

[5]  N. McGlynn.  Thinking fast and slow. , 2014, Australian veterinary journal.

[6]  M. Huijser,et al.  An evaluation of a citizen science data collection program for recording wildlife observations along a highway. , 2014, Journal of environmental management.

[7]  Fabio Massimo Zanzotto.  Human-in-the-loop Artificial Intelligence , 2017, ArXiv.

[8]  E A Baltz,et al.  Achievement of Sustained Net Plasma Heating in a Fusion Experiment with the Optometrist Algorithm , 2017, Scientific Reports.

[9]  Douglas Heaven,et al.  Why deep-learning AIs are so easy to fool , 2019, Nature.

[10]  Pierre Lévy,et al.  Collective Intelligence: Mankind's Emerging World in Cyberspace , 1997 .

[11]  Ellie D'Hondt,et al.  Participatory noise mapping works! An evaluation of participatory sensing as an alternative to standard techniques for environmental monitoring , 2013, Pervasive Mob. Comput..

[12]  Eric Horvitz,et al.  Learning to Complement Humans , 2020, IJCAI.

[13]  Mara Tanelli,et al.  Data-Driven Collaborative Intelligent System for Automatic Activities Monitoring of Wild Animals , 2020, 2020 IEEE International Conference on Human-Machine Systems (ICHMS).

[14]  Petra Klepac,et al.  Contagion! The BBC Four Pandemic - The model behind the documentary. , 2018, Epidemics.

[15]  R. Bonney,et al.  Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy , 2009 .

[17]  Pietro Michelucci,et al.  Human computation requires and enables a new approach to ethical review , 2020, ArXiv.

[18]  C. Lintott,et al.  Galaxy Zoo: morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey , 2008, 0804.4483.

[19]  Joshua B. Tenenbaum,et al.  Human-level concept learning through probabilistic program induction , 2015, Science.

[20]  Yang Song,et al.  The iNaturalist Species Classification and Detection Dataset , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  Z. Popovic,et al.  Increased Diels-Alderase activity through backbone remodeling guided by Foldit players , 2012, Nature Biotechnology.

[22]  Shane Legg,et al.  Deep Reinforcement Learning from Human Preferences , 2017, NIPS.

[23]  P. Russell.  The Global Brain Awakens: Our Next Evolutionary Leap , 1995 .

[24]  Nikhil Prakash,et al.  Conceptualization and Framework of Hybrid Intelligence Systems , 2020, ArXiv.

[25]  Aaron Bauer,et al.  De novo protein design by citizen scientists , 2019, Nature.

[26]  B. Morgan,et al.  Using citizen science butterfly counts to predict species population trends , 2017, Conservation biology : the journal of the Society for Conservation Biology.

[27]  Wolfgang Ziegler,et al.  Swarm Intelligence From Natural To Artificial Systems , 2016 .

[28]  Jody W. Enck,et al.  Can citizen science enhance public understanding of science? , 2016, Public understanding of science.

[29]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[30]  Richard Ernest Bellman,et al.  An Introduction to Artificial Intelligence: Can Computers Think? , 1978 .

[31]  Elizabeth S. Cochran,et al.  The Quake-Catcher Network: Citizen Science Expanding Seismic Horizons , 2009 .

[32]  Ashok N. Srivastava,et al.  Advances in Machine Learning and Data Mining for Astronomy , 2012 .

[33]  Michael Knap,et al.  Classifying snapshots of the doped Hubbard model with machine learning , 2018, Nature Physics.

[34]  Shaeema Zaman Ahmed,et al.  Crowdsourcing human common sense for quantum control , 2020, 2004.03296.

[35]  Tommaso Calarco,et al.  Remote optimization of an ultracold atoms experiment by experts and citizen scientists , 2017, Proceedings of the National Academy of Sciences.

[36]  David Baker,et al.  Foldit Standalone: a video game-derived protein structure manipulation interface using Rosetta , 2017, Bioinform..

[37]  Ingmar H. Riedel-Kruse,et al.  Scientific Discovery Games for Biomedical Research. , 2019, Annual review of biomedical data science.

[38]  Ece Kamar,et al.  Directions in Hybrid Intelligence: Complementing AI Systems with Human Intelligence , 2016, IJCAI.

[39]  M. Haklay.  Citizen Science and Volunteered Geographic Information: Overview and Typology of Participation , 2013 .

[40]  Pietro Michelucci,et al.  The power of crowds , 2016, Science.

[41]  J. Auernhammer,et al.  Design Research in Innovation Management: a pragmatic and human‐centered approach , 2020 .

[42]  Renato Renner,et al.  Discovering physical concepts with neural networks , 2018, Physical review letters.

[43]  Aggelos K. Katsaggelos,et al.  Knowledge Tracing to Model Learning in Online Citizen Science Projects , 2020, IEEE Transactions on Learning Technologies.

[44]  Adrien Treuille,et al.  Predicting protein structures with a multiplayer online game , 2010, Nature.

[45]  J. Tennyson.  The crowd & the cosmos: adventures in the Zooniverse , 2020 .

[46]  Hugo Paredes,et al.  Towards Hybrid Crowd-AI Centered Systems: Developing an Integrated Framework from an Empirical Perspective , 2019, 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC).

[47]  Dominik Dellermann,et al.  Hybrid Intelligence , 2019, Business & Information Systems Engineering.

[48]  Eric Horvitz,et al.  Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure , 2018, HCOMP.

[49]  Gary Marcus,et al.  The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence , 2020, ArXiv.

[50]  Elena Paslaru Bontas Simperl,et al.  An investigation of player motivations in Eyewire, a gamified citizen science project , 2017, Comput. Hum. Behav..

[51]  Melissa Terras,et al.  “Many hands make light work. Many hands together make merry work”: Transcribe Bentham and crowdsourcing manuscript collections , 2014 .

[52]  Michael Heimbinder,et al.  Mapping urban air quality using mobile sampling with low-cost sensors and machine learning in Seoul, South Korea. , 2019, Environment international.

[53]  Wojciech M. Czarnecki,et al.  Grandmaster level in StarCraft II using multi-agent reinforcement learning , 2019, Nature.

[54]  Srinivas C. Turaga,et al.  Space-time wiring specificity supports direction selectivity in the retina , 2014, Nature.

[55]  Felix Motzoi,et al.  Global optimization of quantum dynamics with AlphaZero deep exploration , 2019, npj Quantum Information.

[56]  Chaitanya Joshi,et al.  Human-in-the-loop AI in government: a case study , 2020, IUI.

[57]  David L. Tulloch.  Crowdsourcing geographic knowledge: volunteered geographic information (VGI) in theory and practice , 2014, Int. J. Geogr. Inf. Sci..

[58]  D. Meyer,et al.  Supporting Online Material Materials and Methods Som Text Figs. S1 to S6 References Evidence for a Collective Intelligence Factor in the Performance of Human Groups , 2022 .

[59]  N. Burgess,et al.  The value of indigenous and local knowledge as citizen science , 2018, Citizen Science.

[60]  Chun-Wei Yang,et al.  Applications of artificial intelligence in intelligent manufacturing: a review , 2017, Frontiers of Information Technology & Electronic Engineering.

[61]  Kevin Crowston,et al.  From Conservation to Crowdsourcing: A Typology of Citizen Science , 2011, 2011 44th Hawaii International Conference on System Sciences.

[62]  Prasanna Balaprakash,et al.  Fast domain-aware neural network emulation of a planetary boundary layer parameterization in a numerical weather forecast model , 2019, Geoscientific Model Development.

[63]  Demis Hassabis,et al.  Improved protein structure prediction using potentials from deep learning , 2020, Nature.

[64]  Ulrich Paquet,et al.  Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess , 2020, ArXiv.

[65]  Rich Ormiston,et al.  Noise Reduction in Gravitational-wave Data via Deep Learning , 2020 .

[66]  Finn Danielsen,et al.  Counting what counts: using local knowledge to improve Arctic resource management , 2014 .

[67]  Walter S. Lasecki,et al.  1 On Facilitating Human-Computer Interaction via Hybrid Intelligence Systems , 2019 .

[68]  M. Blanchette,et al.  Phylo: A Citizen Science Approach for Improving Multiple Sequence Alignment , 2012, PloS one.

[69]  Demis Hassabis,et al.  A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play , 2018, Science.

[70]  Michelle J. Wu,et al.  Principles for Predicting RNA Secondary Structure Design Difficulty. , 2016, Journal of molecular biology.

[71]  B. Jack Copeland,et al.  The Broad Conception of Computation , 1997 .

[72]  Barbara Kieslinger,et al.  Supporting emerging forms of citizen science: a plea for diversity, creativity and social innovation , 2016 .

[73]  Minjae Lee,et al.  RNA design rules from a massive open laboratory , 2014, Proceedings of the National Academy of Sciences.

[74]  Yarin Gal,et al.  Galaxy Zoo: Probabilistic Morphology through Bayesian CNNs and Active Learning , 2019, Monthly Notices of the Royal Astronomical Society.

[75]  Antske Fokkens,et al.  A Research Agenda for Hybrid Intelligence: Augmenting Human Intellect With Collaborative, Adaptive, Responsible, and Explainable Artificial Intelligence , 2020, Computer.

[76]  Jordan Raddick,et al.  Galaxy Zoo: Morphological Classification and Citizen Science , 2011, 1104.5513.

[77]  Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts , 2019, CHI PLAY.

[78]  C. Robert.  Superintelligence: Paths, Dangers, Strategies , 2017 .

[79]  Gary Marcus,et al.  Deep Learning: A Critical Appraisal , 2018, ArXiv.

[80]  Ya'akov Gal,et al.  Optimizing Interventions via Offline Policy Evaluation: Studies in Citizen Science , 2018, AAAI.

[81]  Melanie Mitchell,et al.  Ubiquity symposium: Biological Computation , 2011, UBIQ.

[82]  A. Zeilinger,et al.  Automated Search for new Quantum Experiments. , 2015, Physical review letters.

[83]  Z. Popovic,et al.  Crystal structure of a monomeric retroviral protease solved by protein folding game players , 2011, Nature Structural &Molecular Biology.

[84]  Andreas Holzinger,et al.  Interactive machine learning for health informatics: when do we need the human-in-the-loop? , 2016, Brain Informatics.

[85]  Antonio Di Noia,et al.  Mapping atmospheric aerosols with a citizen science network of smartphone spectropolarimeters , 2014 .

[86]  Carla P. Gomes,et al.  Behavior Identification in Two-Stage Games for Incentivizing Citizen Science Exploration , 2016, CP.

[87]  J. V. R. Wintraecken,et al.  The NIAM Information Analysis Method: Theory and Practice , 1990 .