Measuring disagreement in science

Dispute in science is central to the production of new knowledge. Such disputes leave traces in scholarly documents, generally through the form that is taken by citations. Based on the full text of scholarly papers from the Elsevier ScienceDirect database published between 1980 and 2016, this paper develops a methodology for investigating disagreement in science. Several signal phrases of disagreement are tested, and two are used (“contradict” and “conflict”, with the filter phrases “studies” or “results”) to assess the prevalence of disagreement across position within a paper and across disciplines. Results show that disagreement is relatively more common in the introduction and discussion sections of papers, as well as in fields of biomedical sciences, health sciences, social sciences, and humanities.

[1]  J. L. Heilbron,et al.  Leviathan and the air-pump. Hobbes, Boyle, and the experimental life , 1989, Medical History.

[2]  Gianluca Alimonti,et al.  About “scientific consensus on climate change” , 2014 .

[3]  Wolfgang Glänzel,et al.  Bibliometric Evidence for a Hierarchy of the Sciences , 2013, PloS one.

[4]  Chaomei Chen,et al.  A Scalable and Adaptive Method for Finding Semantically Equivalent Cue Words of Uncertainty , 2017, J. Informetrics.

[5]  Christian Catalini,et al.  The incidence and role of negative citations in science , 2015, Proceedings of the National Academy of Sciences.

[6]  Vincent Larivière,et al.  The invariant distribution of references in scientific articles , 2016, J. Assoc. Inf. Sci. Technol..

[7]  Ludo Waltman,et al.  A new methodology for constructing a publication-level classification system of science , 2012, J. Assoc. Inf. Sci. Technol..

[8]  Kevin W. Boyack,et al.  Characterizing in-text citations in scientific articles: A large-scale analysis , 2017, J. Informetrics.

[9]  John H. Evans.  Consensus and knowledge production in an academic field , 2007 .

[10]  Uri Shwed,et al.  The Temporal Structure of Scientific Consensus Formation , 2010, American sociological review.

[11]  M. Moravcsik,et al.  Some Results on the Function and Quality of Citations , 1975 .

[12]  Filippo Radicchi,et al.  In science “there is no bad publicity”: Papers criticized in comments have high scientific impact , 2012, Scientific Reports.

[13]  Ludo Waltman,et al.  Software survey: VOSviewer, a computer program for bibliometric mapping , 2009, Scientometrics.