Toward best practice framing of uncertainty in scientific publications: A review of Water Resources Research abstracts

Uncertainty is recognized as a key issue in water resources research, amongst other sciences. Discussions of uncertainty typically focus on tools and techniques applied within an analysis, e.g. uncertainty quantification and model validation. But uncertainty is also addressed outside the analysis, in writing scientific publications. The language that authors use conveys their perspective of the role of uncertainty when interpreting a claim —what we call here “framing” the uncertainty. This article promotes awareness of uncertainty framing in four ways. 1) It proposes a typology of eighteen uncertainty frames, addressing five questions about uncertainty. 2) It describes the context in which uncertainty framing occurs. This is an interdisciplinary topic, involving philosophy of science, science studies, linguistics, rhetoric, and argumentation. 3) We analyze the use of uncertainty frames in a sample of 177 abstracts from the Water Resources Research journal in 2015. This helped develop and tentatively verify the typology, and provides a snapshot of current practice. 4) Provocative recommendations promote adjustments for a more influential, dynamic science. Current practice in uncertainty framing might be described as carefully-considered incremental science. In addition to uncertainty quantification and degree of belief (present in ∼5% of abstracts), uncertainty is addressed by a combination of limiting scope, deferring to further work (∼25%) and indicating evidence is sufficient (∼40%) – or uncertainty is completely ignored (∼8%). There is a need for public debate within our discipline to decide in what context different uncertainty frames are appropriate. Uncertainty framing cannot remain a hidden practice evaluated only by lone reviewers.

[1]  Claudia Pahl-Wostl,et al.  Assessing Framing of Uncertainties in Water Management Practice , 2009 .

[2]  Trevor Pinch,et al.  Towards an Analysis of Scientific Observation: The Externality and Evidential Significance of Observational Reports in Physics , 1985 .

[3]  John P. Norton,et al.  To what extent, and how, might uncertainty be defined? Comments engendered by "Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support": Walker et al., Integrated Assessment 4: 1, 2003 , 2006 .

[4]  V. Klemeš,et al.  Dilettantism in hydrology: Transition or destiny? , 1986 .

[5]  James C. I. Dooge,et al.  Walter Langbein and the emergence of scientific hydrology , 1996 .

[6]  Soroosh Sorooshian,et al.  Correction to “Toward improved identifiability of hydrologic model parameters: The information content of experimental data” , 2003 .

[7]  Keith Beven,et al.  Calibration of watershed models , 2003 .

[8]  András Bárdossy,et al.  The Court of Miracles of Hydrology: can failure stories contribute to hydrological science? , 2010 .

[9]  Richard P. Hooper,et al.  Hydrology: The interdisciplinary science of water , 2015 .

[10]  W. Walker,et al.  Defining Uncertainty: A Conceptual Basis for Uncertainty Management in Model-Based Decision Support , 2003 .

[11]  Max Henrion,et al.  Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis , 1990 .

[12]  Hoshin Vijai Gupta,et al.  Debates—the future of hydrological sciences: A (common) path forward? Using models and data to learn: A systems theoretic perspective on the future of hydrological science , 2014 .

[13]  Joseph H. A. Guillaume,et al.  An uncertain future, deep uncertainty, scenarios, robustness and adaptation: How do they fit together? , 2016, Environ. Model. Softw..

[14]  Brian Street,et al.  "Hidden" Features of Academic Paper Writing , 2009 .

[15]  Greg Myers,et al.  Texts as Knowledge Claims: The Social Construction of Two Biology Articles , 1985 .

[16]  Joseph H. A. Guillaume,et al.  Fit for purpose? Building and evaluating a fast, integrated model for exploring water policy pathways , 2014, Environ. Model. Softw..

[17]  Helen M. Regan,et al.  A TAXONOMY AND TREATMENT OF UNCERTAINTY FOR ECOLOGY AND CONSERVATION BIOLOGY , 2002 .

[18]  James D. Brown Knowledge, uncertainty and physical geography: towards the development of methodologies for questioning belief , 2004 .

[19]  Phan Minh Dung,et al.  On the Acceptability of Arguments and its Fundamental Role in Nonmonotonic Reasoning, Logic Programming and n-Person Games , 1995, Artif. Intell..

[20]  John L. Pollock,et al.  Defeasible Reasoning , 2020, Synthese Library.

[21]  Z. Fang Scientific literacy: A systemic functional linguistics perspective , 2005 .

[22]  Claudia Pahl-Wostl,et al.  Toward a Relational Concept of Uncertainty: about Knowing Too Little, Knowing Too Differently, and Accepting Not to Know , 2008 .

[23]  C. Tiedeman,et al.  Effective Groundwater Model Calibration , 2007 .

[24]  Peter Crompton,et al.  Hedging in academic writing: Some theoretical problems , 1997 .

[25]  W. Parker II—Wendy S. Parker: Confirmation and adequacy-for-Purpose in Climate Modelling , 2009 .

[26]  Joseph H. A. Guillaume,et al.  Characterising performance of environmental models , 2013, Environ. Model. Softw..

[27]  Ming Ye,et al.  Towards a comprehensive assessment of model structural adequacy , 2012 .

[28]  M. B. Beck,et al.  Water quality modeling: A review of the analysis of uncertainty , 1987 .

[29]  Mark E. Borsuk,et al.  Does high forecast uncertainty preclude effective decision support? , 2005, Environ. Model. Softw..

[30]  Soroosh Sorooshian,et al.  Toward improved calibration of hydrologic models: Multiple and noncommensurable measures of information , 1998 .

[31]  F. Pappenberger,et al.  Ignorance is bliss: Or seven reasons not to use uncertainty analysis , 2006 .

[32]  Sampo Pyysalo,et al.  brat: a Web-based Tool for NLP-Assisted Text Annotation , 2012, EACL.

[33]  Gunther Kaltenböck,et al.  New approaches to hedging , 2010 .

[34]  Keith Beven,et al.  A guide to good practice in modeling semantics for authors and referees , 2013 .

[35]  H. Künsch,et al.  Practical identifiability analysis of large environmental simulation models , 2001 .

[36]  R. Lipshitz,et al.  Coping with Uncertainty: A Naturalistic Decision-Making Analysis , 1997 .

[37]  L. Shawn Matott,et al.  Evaluating uncertainty in integrated environmental models: A review of concepts and tools , 2009 .

[38]  Peter Reichert,et al.  The conceptual foundation of environmental decision support. , 2015, Journal of environmental management.

[39]  Robert J Lempert,et al.  A new decision sciences for complex systems , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[40]  Steve Bankes,et al.  Exploratory Modeling for Policy Analysis , 1993, Oper. Res..

[41]  C. Coffin,et al.  Academic literacies and systemic functional linguistics: how do they relate? , 2012 .

[42]  B. Latour,et al.  Laboratory Life: The Construction of Scientific Facts , 1979 .

[43]  W. James Potter,et al.  Rethinking validity and reliability in content analysis , 1999 .

[44]  Wolfgang Ziegler,et al.  An Introduction To Reasoning , 2016 .

[45]  C. Tiedeman,et al.  Effective Groundwater Model Calibration: With Analysis of Data, Sensitivities, Predictions, and Uncertainty , 2007 .

[46]  Hoshin Vijai Gupta,et al.  The quantity and quality of information in hydrologic models , 2015 .

[47]  Ken Hyland,et al.  Stance and engagement: a model of interaction in academic discourse , 2005 .

[48]  James S. Risbey,et al.  Sources of knowledge and ignorance in climate research , 2011 .

[49]  Joseph H. A. Guillaume,et al.  Methods for exploring uncertainty in groundwater management predictions , 2016 .

[50]  M. Hufschmidt The role of universities in water resources education: The social sciences , 1967 .

[51]  Joseph H. A. Guillaume,et al.  Communicating uncertainty: design patterns for framing model results in scientific publications , 2015 .

[52]  Robert F. Boruch,et al.  Standards of Evidence: Criteria for Efficacy, Effectiveness and Dissemination , 2005, Prevention Science.

[53]  Warren E. Walker,et al.  Adapt or Perish: A Review of Planning Approaches for Adaptation under Deep Uncertainty , 2013 .

[54]  Hoshin Vijai Gupta,et al.  Model identification for hydrological forecasting under uncertainty , 2005 .

[55]  Peter A. Vanrolleghem,et al.  Uncertainty in the environmental modelling process - A framework and guidance , 2007, Environ. Model. Softw..

[56]  H. Simon,et al.  Rational choice and the structure of the environment. , 1956, Psychological review.

[57]  Alexander Zahar,et al.  Shaping written knowledge: the genre and activity of the experimental article in science , 1991, Medical History.

[58]  T. Jick Mixing Qualitative and Quantitative Methods: Triangulation in Action. , 1979 .

[59]  Warren E. Walker,et al.  Response to “To what extent, and how, might uncertainty be defined” by Norton, Brown, and Mysiak , 2006 .

[60]  Thomas Princen,et al.  Principles for Sustainability: From Cooperation and Efficiency to Sufficiency , 2003, Global Environmental Politics.

[61]  Soroosh Sorooshian,et al.  Toward improved identifiability of hydrologic model parameters: The information content of experimental data , 2002 .

[62]  Wim J. de Lange,et al.  Uncertainty Matters: Computer Models at the Science–Policy Interface , 2007 .

[63]  G. Gilbert,et al.  The Transformation of Research Findings into Scientific Knowledge , 1976 .

[64]  Fabrizio Douglas Macagno,et al.  Classifying the Patterns of Natural Arguments , 2015 .

[65]  J. Swales,et al.  Genre identification and communicative purpose: A problem and a possible solution , 2001 .

[66]  Wiebe E. Bijker,et al.  Science in action : how to follow scientists and engineers through society , 1989 .

[67]  Gregory J. Kelly,et al.  How Students Argue Scientific Claims: A Rhetorical‐Semantic Analysis , 2003 .

[68]  Murugesu Sivapalan,et al.  Debates—Perspectives on socio‐hydrology: Changing water systems and the “tyranny of small problems”—Socio‐hydrology , 2015 .

[69]  G. A. Bradshaw,et al.  Conservation Ecology: Uncertainty as Information: Narrowing the Science-policy Gap , 2008 .

[70]  Ken Hyland,et al.  Genre and academic writing in the disciplines , 2008, Language Teaching.

[71]  Cajo J. F. ter Braak,et al.  Treatment of input uncertainty in hydrologic modeling: Doing hydrology backward with Markov chain Monte Carlo simulation , 2008 .

[72]  B. Latour,et al.  Laboratory Life: The Social Construction of Scientific Facts , 1983 .