A Review on the Applications of Crowdsourcing in Human Pathology

The advent of the digital pathology has introduced new avenues of diagnostic medicine. Among them, crowdsourcing has attracted researchers' attention in the recent years, allowing them to engage thousands of untrained individuals in research and diagnosis. While there exist several articles in this regard, prior works have not collectively documented them. We, therefore, aim to review the applications of crowdsourcing in human pathology in a semi-systematic manner. We first, introduce a novel method to do a systematic search of the literature. Utilizing this method, we, then, collect hundreds of articles and screen them against a predefined set of criteria. Furthermore, we crowdsource part of the screening process, to examine another potential application of crowdsourcing. Finally, we review the selected articles and characterize the prior uses of crowdsourcing in pathology.

[1]  Martin H. Levinson.  Longitude: The True Story of a Lone Genius Who Solved the Greatest Scientific Problem of His Time , 1997 .

[2]  Daniel Gautheret,et al.  Metagenome Annotation Using a Distributed Grid of Undergraduate Students , 2008, PLoS biology.

[3]  Miguel Angel Luengo-Oroz,et al.  Crowdsourcing Malaria Parasite Quantification: An Online Game for Analyzing Images of Infected Thick Blood Smears , 2012, Journal of medical Internet research.

[4]  J. M. Crawford,et al.  Pathologist workforce in the United States: I. Development of a predictive model to examine factors influencing supply. , 2013, Archives of pathology & laboratory medicine.

[5]  Andrew H. Beck,et al.  Crowdsourcing scoring of immunohistochemistry images: Evaluating Performance of the Crowd and an Automated Computational Method , 2016, Scientific Reports.

[6]  Adrien Treuille,et al.  Predicting protein structures with a multiplayer online game , 2010, Nature.

[7]  C. Lintott,et al.  Galaxy Zoo 1: data release of morphological classifications for nearly 900 000 galaxies , 2010, 1007.3265.

[8]  Diogo M. Camacho,et al.  Wisdom of crowds for robust gene network inference , 2012, Nature Methods.

[9]  Liron Pantanowitz,et al.  Review of advanced imaging techniques , 2012, Journal of pathology informatics.

[10]  Leigh-Anne McDuffus,et al.  Crowdsourcing for translational research: analysis of biomarker expression using cancer microarrays , 2016, British Journal of Cancer.

[11]  Laura A. Dabbish,et al.  Designing games with a purpose , 2008, CACM.

[12]  Steve Feng,et al.  Distributed Medical Image Analysis and Diagnosis through Crowd-Sourced Games: A Malaria Case Study , 2012, PloS one.

[13]  Hayit Greenspan,et al.  Deep learning with non-medical training used for chest pathology identification , 2015, Medical Imaging.

[14]  Tomoharu Kiyuna,et al.  Pathological Diagnosis of Gastric Cancers with a Novel Computerized Analysis System , 2017, Journal of pathology informatics.

[15]  Minjae Lee,et al.  RNA design rules from a massive open laboratory , 2014, Proceedings of the National Academy of Sciences.

[16]  Zachary F. Meisel,et al.  Crowdsourcing—Harnessing the Masses to Advance Health and Medicine, a Systematic Review , 2013, Journal of General Internal Medicine.

[17]  Emad A. Mohammed,et al.  Peripheral blood smear image analysis: A comprehensive review , 2014, Journal of pathology informatics.

[18]  Ghassan Hamarneh,et al.  Clinically-inspired automatic classification of ovarian carcinoma subtypes , 2016, Journal of pathology informatics.

[19]  Benjamin M. Good,et al.  Crowdsourcing for bioinformatics , 2013, Bioinform..

[20]  Andrew Janowczyk,et al.  Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases , 2016, Journal of pathology informatics.

[21]  Steve Feng,et al.  Crowd-sourced BioGames: managing the big data problem for next-generation lab-on-a-chip platforms. , 2012, Lab on a chip.

[22]  C. Shahabi,et al.  On Acquisition and Analysis of Visual Data for Crowdsourcing Disaster Response , 2016 .

[23]  Cyrus Shahabi,et al.  An entropy-based framework for efficient post-disaster assessment based on crowdsourced data , 2016, EM-GIS.

[24]  Optical Biopsy of Bladder Cancer Using Crowd-Sourced Assessment. , 2016, JAMA surgery.

[25]  Manuel Blum,et al.  reCAPTCHA: Human-Based Character Recognition via Web Security Measures , 2008, Science.

[26]  Bin Liu,et al.  Crowdsourcing the General Public for Large Scale Molecular Pathology Studies in Cancer , 2015, EBioMedicine.