Crowdsourcing in medical research: concepts and applications

Crowdsourcing shifts medical research from a closed environment to an open collaboration between the public and researchers. We define crowdsourcing as an approach to problem solving which involves an organization having a large group attempt to solve a problem or part of a problem, then sharing solutions. Crowdsourcing allows large groups of individuals to participate in medical research through innovation challenges, hackathons, and related activities. The purpose of this literature review is to examine the definition, concepts, and applications of crowdsourcing in medicine. This multi-disciplinary review defines crowdsourcing for medicine, identifies conceptual antecedents (collective intelligence and open source models), and explores implications of the approach. Several critiques of crowdsourcing are also examined. Although several crowdsourcing definitions exist, there are two essential elements: (1) having a large group of individuals, including those with skills and those without skills, propose potential solutions; (2) sharing solutions through implementation or open access materials. The public can be a central force in contributing to formative, pre-clinical, and clinical research. A growing evidence base suggests that crowdsourcing in medicine can result in high-quality outcomes, broad community engagement, and more open science.

[1]  Bernard H. Munos,et al.  Can open-source R&D reinvigorate drug research? , 2006, Nature Reviews Drug Discovery.

[2]  Zachary F. Meisel,et al.  Crowdsourcing—Harnessing the Masses to Advance Health and Medicine, a Systematic Review , 2013, Journal of General Internal Medicine.

[3]  Allison Mathews,et al.  HIV cure research community engagement in North Carolina: a mixed-methods evaluation of a crowdsourcing contest , 2017, Journal of virus eradication.

[4]  Sarah Bendall,et al.  Online and Social Media Suicide Prevention Interventions for Young People: A Focus on Implementation and Moderation. , 2016, Journal of the Canadian Academy of Child and Adolescent Psychiatry = Journal de l'Academie canadienne de psychiatrie de l'enfant et de l'adolescent.

[5]  Pierre Lévy,et al.  Collective Intelligence: Mankind's Emerging World in Cyberspace , 1997 .

[6]  P. English,et al.  From Crowdsourcing to Extreme Citizen Science: Participatory Research for Environmental Health. , 2018, Annual review of public health.

[7]  Byron C. Wallace,et al.  An exploration of crowdsourcing citation screening for systematic reviews , 2017, Research synthesis methods.

[8]  Daniel L. Shaw Is Open Science the Future of Drug Development? , 2017, The Yale journal of biology and medicine.

[9]  J. Clarke,et al.  Medicine , 1907, Bristol medico-chirurgical journal.

[10]  Daren C. Brabham,et al.  Crowdsourcing applications for public health. , 2014, American journal of preventive medicine.

[11]  Ben Kirshner,et al.  Productive Tensions in Youth Participatory Action Research , 2010, Teachers College Record: The Voice of Scholarship in Education.

[12]  Weiming Tang,et al.  Creative Contributory Contests to Spur Innovation in Sexual Health: 2 Cases and a Guide for Implementation. , 2015, Sexually transmitted diseases.

[13]  Allison Mathews,et al.  Crowdsourcing and community engagement: a qualitative analysis of the 2BeatHIV contest , 2018, Journal of virus eradication.

[14]  Li Liu,et al.  A Crowdsourcing Approach to Developing and Assessing Prediction Algorithms for AML Prognosis , 2016, PLoS Comput. Biol..

[15]  Weiming Tang,et al.  Community participation in a Lancet Healthy Cities in China Commission. , 2018, The Lancet. Planetary health.

[16]  M. Hudgens,et al.  Reimagining Health Communication: A Noninferiority Randomized Controlled Trial of Crowdsourced Intervention in China , 2019, Sexually transmitted diseases.

[17]  Christine Årdal,et al.  Open Source Drug Discovery in Practice: A Case Study , 2012, PLoS neglected tropical diseases.

[18]  ALLAN FERGUSON,et al.  World Brain , 1938, Nature.

[19]  Matthew H Todd,et al.  Addressing the most neglected diseases through an open research model: The discovery of fenarimols as novel drug candidates for eumycetoma , 2018, bioRxiv.

[20]  Alfonso Mendoza,et al.  Fueling Open-Source Drug Discovery: 177 Small-Molecule Leads against Tuberculosis , 2013, ChemMedChem.

[21]  Roberta Lanzillo,et al.  Social Media and Multiple Sclerosis in the Posttruth Age , 2017, Interactive journal of medical research.

[22]  Mathew D. Sorensen,et al.  Crowdsourcing in Surgical Skills Acquisition: A Developing Technology in Surgical Education. , 2017, Journal of graduate medical education.

[23]  Pallava Bagla Science in India. Crowd-sourcing drug discovery. , 2012, Science.

[24]  Hans Järnbert-Pettersson,et al.  Mobile-phone dispatch of laypersons for CPR in out-of-hospital cardiac arrest. , 2015, The New England journal of medicine.

[25]  B. Feldman,et al.  Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. , 2014, Journal of clinical epidemiology.

[26]  Emily J Ozer,et al.  Youth-Led Participatory Action Research: Developmental and Equity Perspectives. , 2016, Advances in child development and behavior.

[27]  Rosanna Peeling,et al.  Crowdsourcing HIV Test Promotion Videos: A Noninferiority Randomized Controlled Trial in China. , 2016, Clinical infectious diseases : an official publication of the Infectious Diseases Society of America.

[28]  Daren C. Brabham Crowdsourcing as a Model for Problem Solving , 2008 .

[29]  Allison Mathews,et al.  Ethical Concerns of and Risk Mitigation Strategies for Crowdsourcing Contests and Innovation Challenges: Scoping Review , 2018, Journal of medical Internet research.

[30]  D. Meyer,et al.  Supporting Online Material Materials and Methods Som Text Figs. S1 to S6 References Evidence for a Collective Intelligence Factor in the Performance of Human Groups , 2022 .

[31]  A. Leiter,et al.  Use of crowdsourcing for cancer clinical trial development. , 2014, Journal of the National Cancer Institute.

[32]  Ronald Maier,et al.  Convergence of Crowdsourcing Ideas: A Cognitive Load perspective , 2017, ICIS.

[33]  Allison Mathews,et al.  Crowdsourcing in health and medical research: a systematic review , 2020, Infectious Diseases of Poverty.

[34]  Shadi Albarqouni,et al.  AggNet : Deep Learning From Crowds for Mitosis Detection in Breast Cancer Histology Images , 2016 .

[35]  Christine Årdal,et al.  An Open Source Business Model for Malaria , 2015, PloS one.

[36]  Jaijit Bhattacharya,et al.  Open source drug discovery--a new paradigm of collaborative research in tuberculosis drug development. , 2011, Tuberculosis.

[37]  K. Ladha The Condorcet Jury Theorem, Free Speech and Correlated Votes , 1992 .

[38]  Wei Zhang,et al.  Innovation contests to promote sexual health in china: a qualitative evaluation , 2017, BMC Public Health.

[39]  Jayant V. Narlikar Science in India , 1946, Nature.

[40]  J. Hahn Victims Of Groupthink A Psychological Study Of Foreign Policy Decisions And Fiascoes , 2016 .

[41]  Kim Bure,et al.  Open Access Could Transform Drug Discovery: A Case Study of JQ1 , 2016, Expert opinion on drug discovery.

[42]  Michael G Hudgens,et al.  Crowdsourcing to expand HIV testing among men who have sex with men in China: A closed cohort stepped wedge cluster randomized controlled trial , 2018, PLoS medicine.

[43]  L. Davis,et al.  Drug repositioning in SLE: crowd-sourcing, literature-mining and Big Data analysis , 2016, Lupus.

[44]  Philippe Ravaud,et al.  Mapping of Crowdsourcing in Health: Systematic Review , 2018, Journal of medical Internet research.

[45]  Junhua Li,et al.  Open-source genomic analysis of Shiga-toxin-producing E. coli O104:H4. , 2011, The New England journal of medicine.

[46]  Yu Cheng,et al.  A crowdsourced intervention to promote hepatitis B and C testing among men who have sex with men in China: study protocol for a nationwide online randomized controlled trial , 2018, BMC Infectious Diseases.

[47]  Klaus Gottlieb,et al.  Voting for Image Scoring and Assessment (VISA) - theory and application of a 2 + 1 reader algorithm to improve accuracy of imaging endpoints in clinical trials , 2015, BMC Medical Imaging.

[48]  Kerri Wazny,et al.  “Crowdsourcing” ten years in: A review , 2017, Journal of global health.

[49]  P. Mink,et al.  Implementing a Community-Driven Research Partnership: The Backyard Initiative Community Health Survey Methods and Approach , 2017, Progress in community health partnerships : research, education, and action.

[50]  Aman Verma,et al.  Media content about vaccines in the United States and Canada, 2012-2014: An analysis using data from the Vaccine Sentimeter. , 2016, Vaccine.

[51]  Stefan M. Herzog,et al.  Boosting medical diagnostics by pooling independent judgments , 2016, Proceedings of the National Academy of Sciences.

[52]  Ingrid Holsen,et al.  Innovative Approaches to Promoting Positive Youth Development in Diverse Contexts: Novel Applications of Participatory Research and New Technologies , 2016 .

[53]  J. Banks,et al.  Information Aggregation, Rationality, and the Condorcet Jury Theorem , 1996, American Political Science Review.

[54]  Alice Zhang,et al.  What Would an HIV Cure Mean to You? Qualitative Analysis from a Crowdsourcing Contest in Guangzhou, China. , 2017, AIDS research and human retroviruses.

[55]  Meredith Minkler,et al.  Community-Based Participatory Research for Health , 2002 .

[56]  Joseph D. Tucker,et al.  Reimagining Health Communication: A Non-Inferiority Randomized Controlled Trial of Crowdsourcing in China , 2017 .

[57]  Michael Rosemann,et al.  Crowdsourcing Information Systems - A Systems Theory Perspective , 2011 .

[58]  H. Chaiklin,et al.  The Myth of the Madding Crowd , 1992 .

[59]  Weiming Tang,et al.  Crowdsourcing health communication about condom use in men who have sex with men in China: a randomised controlled trial , 2016, The Lancet.

[60]  Allison Mathews,et al.  Systematic review of innovation design contests for health: spurring innovation and mass engagement , 2017, BMJ Innovations.

[61]  Edmund A. Mennis The Wisdom of Crowds: Why the Many Are Smarter than the Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations , 2006 .

[62]  Elena Losina,et al.  Randomized Controlled Trial of an Educational Intervention Using an Online Risk Calculator for Knee Osteoarthritis: Effect on Risk Perception , 2017, Arthritis care & research.

[63]  Paul Hewson,et al.  Recruitment to Online Therapies for Depression: Pilot Cluster Randomized Controlled Trial , 2013, Journal of medical Internet research.

[64]  Wolfgang Gaissmaier,et al.  Diagnostic performance by medical students working individually or in teams. , 2015, JAMA.

[65]  Nicolas de Condorcet Essai Sur L'Application de L'Analyse a la Probabilite Des Decisions Rendues a la Pluralite Des Voix , 2009 .

[66]  Georg von Krogh,et al.  Open Source Software and the "Private-Collective" Innovation Model: Issues for Organization Science , 2003, Organ. Sci..

[67]  Charles Mackay Memoirs of Extraordinary Popular Delusions and the Madness of Crowds , 2019 .

[68]  Henry Chesbrough,et al.  Open Innovation: The New Imperative for Creating and Profiting from Technology , 2003 .

[69]  Michael W. Weiner,et al.  Crowdsourced estimation of cognitive decline and resilience in Alzheimer's disease , 2016, Alzheimer's & Dementia.

[70]  S. Friend,et al.  Crowdsourcing biomedical research: leveraging communities as innovation engines , 2016, Nature Reviews Genetics.

[71]  Kerri Wazny “Crowdsourcing” ten years in: A review , 2017, Journal of Global Health.

[72]  Pierre Tufféry,et al.  Accessing external innovation in drug discovery and development , 2015, Expert opinion on drug discovery.

[73]  Hannah Coaker,et al.  Neglected tropical disease research: rethinking the drug discovery model. , 2015, Future medicinal chemistry.

[74]  Chris Smith,et al.  Community‐Driven Research Agenda to Reduce Health Disparities , 2015, Clinical and translational science.

[75]  Michael G Hudgens,et al.  Generalisability of an online randomised controlled trial: an empirical analysis , 2017, Journal of Epidemiology & Community Health.

[76]  Katie Richards-Schuster,et al.  Revolutionizing education: Youth participatory action research in motion , 2010 .

[77]  David S Friedman,et al.  Crowdsourcing and Automated Retinal Image Analysis for Diabetic Retinopathy , 2017, Current Diabetes Reports.

[78]  Stefan M. Herzog,et al.  The Potential of Collective Intelligence in Emergency Medicine: Pooling Medical Students’ Independent Decisions Improves Diagnostic Performance , 2017, Medical decision making : an international journal of the Society for Medical Decision Making.

[79]  Thomas Yu,et al.  Prediction of overall survival for patients with metastatic castration-resistant prostate cancer: development of a prognostic model through a crowdsourced challenge with open clinical trial data. , 2017, The Lancet. Oncology.

[80]  Tom Koch,et al.  The Condorcet's Jury Theorem in a Bioethical Context: The Dynamics of Group Decision Making , 2000 .

[81]  Andrew R. Scott Chemical probes: A shared toolbox , 2016, Nature.

[82]  Weiming Tang,et al.  Reimagining Health Communication: A Non-Inferiority Randomized Controlled Trial of Crowdsourced intervention in China. , 2019, Sexually transmitted diseases.

[83]  John S Brownstein,et al.  Measuring vaccine confidence: analysis of data obtained by a media surveillance system used to analyse public concerns about vaccines. , 2013, The Lancet. Infectious diseases.

[84]  J Weigelt,et al.  Towards new business models for R&D for novel antibiotics. , 2011, Drug resistance updates : reviews and commentaries in antimicrobial and anticancer chemotherapy.

[85]  Matthew A Cooper,et al.  Institutional profile: Community for Open Antimicrobial Drug Discovery – crowdsourcing new antibiotics and antifungals , 2017, Future science OA.

[86]  Ralf H. J. M. Kurvers,et al.  Collective Intelligence Meets Medical Decision-Making: The Collective Outperforms the Best Radiologist , 2015, PloS one.

[87]  Ayesha N Khalid,et al.  The Yale Center for Biomedical Innovation and Technology (CBIT): One Model to Accelerate Impact From Academic Health Care Innovation. , 2019, Academic medicine : journal of the Association of American Medical Colleges.

[88]  B. Munos,et al.  Can Open‐Source Drug R&D Repower Pharmaceutical Innovation? , 2010, Clinical pharmacology and therapeutics.

[89]  H. Bradbury The SAGE Handbook of Action Research , 2007 .

[90]  Jeremy N. Burrows,et al.  The Open Access Malaria Box: A Drug Discovery Catalyst for Neglected Diseases , 2013, PloS one.

[91]  Antony Stephen Reid Manstead,et al.  The Blackwell Encyclopedia of Social Psychology , 1996 .