Scientific Auditing Firms

The “crisis of reproducibility” has been a significant source of controversy, heated debate, and calls for reform to institutional science in recent years. As a long-term solution to address both the present crisis and future obstacles, I propose the creation of a new form of research organization whose purpose would be to conduct random audits of the scientific literature. I suggest that data analytics of a digitized scientific corpus may play a critical role in allowing broadly educated scientists to identify linchpin results to investigate in further detail across all disciplines. I argue that a simple “mock” trial run of a simplified auditing firm consisting of several researchers over a short time period would provide valuable insight into the feasibility of this proposal.

[1]  Ying Ding,et al.  The distribution of references across texts: Some implications for citation analysis , 2013, J. Informetrics.

[2]  Sergi Valverde,et al.  Topology and evolution of technology innovation networks. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[3]  W. Gunn Reproducibility: Fraud is not the big problem , 2014, Nature.

[4]  Jonathan Knight,et al.  Journals under pressure: Publish, and be damned... , 2002, Nature.

[5]  Sander Greenland,et al.  Why Most Published Research Findings Are False: Problems in the Analysis , 2007, PLoS medicine.

[6]  Daniel Lemire,et al.  Measuring academic influence: Not all citations are equal , 2015, J. Assoc. Inf. Sci. Technol..

[7]  Harold Varmus,et al.  Rescuing US biomedical research from its systemic flaws , 2014, Proceedings of the National Academy of Sciences.

[8]  J. Mayet,et al.  Meta-analysis of secure randomised controlled trials of β-blockade to prevent perioperative death in non-cardiac surgery , 2013, Heart.

[9]  A. Millmow and be Damned , 1993 .

[10]  Ricard V. Solé,et al.  The evolutionary ecology of technological innovations , 2013, Complex..

[11]  Jeffrey T. Hancock,et al.  Linguistic Obfuscation in Fraudulent Science , 2016 .

[12]  J. Brooks Why most published research findings are false: Ioannidis JP, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece , 2008 .

[13]  D. Cyranoski,et al.  Korean scandal will have global fallout , 2005, Nature.

[14]  Guo Zhang,et al.  Productivity and influence in bioinformatics: A bibliometric analysis using PubMed central , 2014, J. Assoc. Inf. Sci. Technol..

[15]  Ying Ding,et al.  Topic-based PageRank on author cocitation networks , 2011, J. Assoc. Inf. Sci. Technol..

[16]  Ying Ding,et al.  Applying weighted PageRank to author citation networks , 2011, J. Assoc. Inf. Sci. Technol..

[17]  Bernard Gress,et al.  Properties of the USPTO patent citation network: 1963-2002 , 2010 .

[18]  Jiancheng Guan,et al.  A bibliometric study of service innovation research: based on complex network analysis , 2012, Scientometrics.

[19]  F. Prinz,et al.  Believe it or not: how much can we rely on published data on potential drug targets? , 2011, Nature Reviews Drug Discovery.