Why are we not evaluating multiple competing hypotheses in ecology and evolution?

The use of multiple working hypotheses to gain strong inference is widely promoted as a means to enhance the effectiveness of scientific investigation. Only 21 of 100 randomly selected studies from the ecological and evolutionary literature tested more than one hypothesis and only eight tested more than two hypotheses. The surprising rarity of application of multiple working hypotheses suggests that this gap between theory and practice might reflect some fundamental issues. Here, we identify several intellectual and practical barriers that discourage us from using multiple hypotheses in our scientific investigation. While scientists have developed a number of ways to avoid biases, such as the use of double-blind controls, we suspect that few scientists are fully aware of the potential influence of cognitive bias on their decisions and they have not yet adopted many techniques available to overcome intellectual and practical barriers in order to improve scientific investigation.

[1]  Robert H. Gardner,et al.  Neutral Models: Useful Tools for Understanding Landscape Patterns , 1997 .

[2]  F. Hecht “Surely you're joking, Mr. Feynman!” Adventures of a curious character , 1986 .

[3]  Jerald B. Johnson,et al.  Model selection in ecology and evolution. , 2004, Trends in ecology & evolution.

[4]  K. Laland,et al.  THE NICHE CONSTRUCTION PERSPECTIVE: A CRITICAL APPRAISAL , 2014, Evolution; international journal of organic evolution.

[5]  Leif D. Nelson,et al.  Data from Paper “False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant” , 2014 .

[6]  Jonathan St. B. T. Evans,et al.  Questions and challenges for the new psychology of reasoning , 2012 .

[7]  Michael E. Gorman,et al.  Scientific and Technological Thinking , 2006 .

[8]  J. Ioannidis Why Most Published Research Findings Are False , 2019, CHANCE.

[9]  T. Kuhn,et al.  The Structure of Scientific Revolutions. , 1964 .

[10]  Aaron Roodman,et al.  Blind Analysis in Nuclear and Particle Physics , 2005 .

[11]  S. Hubbell The Unified Neutral Theory of Biodiversity and Biogeography (MPB-32) , 2011 .

[12]  J. Platt Strong Inference , 2007 .

[13]  Lisa Jardine,et al.  The New Organon , 2008 .

[14]  T. C. Chamberlin The Method of Multiple Working Hypotheses , 1931, The Journal of Geology.

[15]  D. Hull,et al.  Science as a Process: An Evolutionary Account of the Social and Conceptual Development of Science, David L. Hull. 1988. The University of Chicago Press, Chicago, IL. 608 pages. ISBN: 0-226-35060-4. $39.95 , 1989 .

[16]  Craig Loehle,et al.  Complexity and the problem of ill-posed questions in ecology , 2011 .

[17]  R. Dehaan,et al.  Teaching Creative Science Thinking , 2011, Science.

[18]  J. Stegenga Evidence in biology and the conditions of success , 2013 .

[19]  A. Barabasi,et al.  Quantifying the evolution of individual scientific impact , 2016, Science.

[20]  J. I The Design of Experiments , 1936, Nature.

[21]  Hanna Kokko,et al.  The evolution of superstitious and superstition-like behaviour , 2009, Proceedings of the Royal Society B: Biological Sciences.

[22]  R. Nickerson Confirmation Bias: A Ubiquitous Phenomenon in Many Guises , 1998 .

[23]  I. Mitroff Norms and Counter-Norms in a Select Group of the Apollo Moon Scientists: A Case Study of the Ambivalence of Scientists , 1974 .

[24]  Steve Kelling,et al.  Crowdsourcing Meets Ecology: Hemisphere-Wide Spatiotemporal Species Distribution Models , 2014, AI Mag..

[25]  Clifford R. Mynatt,et al.  On Scientific Thinking , 1981 .

[26]  J. Ioannidis Why Most Published Research Findings Are False , 2005, PLoS medicine.

[27]  J. Platt Strong Inference: Certain systematic methods of scientific thinking may produce much more rapid progress than others. , 1964, Science.

[28]  T. C. CHAMBERLIN The Method of Multiple Working Hypotheses , 1931, The Journal of Geology.

[29]  H. Lacey Is Science Value Free?: Values and Scientific Understanding , 2004 .

[30]  Peter Kúš Experimental , 2019, Springer Theses.

[31]  R. Nuzzo How scientists fool themselves – and how they can stop , 2015, Nature.

[32]  Saul Perlmutter,et al.  Blind analysis: Hide results to seek the truth , 2015, Nature.

[33]  S. Hubbell,et al.  The unified neutral theory of biodiversity and biogeography at age ten. , 2011, Trends in ecology & evolution.

[34]  D. Freedman A Note on Screening Regression Equations , 1983 .

[35]  C Loehle,et al.  Hypothesis Testing in Ecology: Psychological Aspects and the Importance of Theory Maturation , 1987, The Quarterly Review of Biology.

[36]  Wim De Neys,et al.  Belief inhibition during thinking: Not always winning but at least taking part , 2009, Cognition.

[37]  Sam Cocks,et al.  The Science of the Struggle for Existence: On the Foundations of Ecology , 2005 .

[38]  J. Stevenson,et al.  Defenders of the truth: The battle for science in the sociobiology debate and beyond , 2001 .

[39]  R. Hilborn,et al.  On inference in ecology and evolutionary biology: the problem of multiple causes , 1982, Acta biotheoretica.

[40]  James H Brown,et al.  Multiple ecological pathways to extinction in mammals , 2009, Proceedings of the National Academy of Sciences.

[41]  David R. Anderson,et al.  AIC model selection and multimodel inference in behavioral ecology: some background, observations, and comparisons , 2011, Behavioral Ecology and Sociobiology.

[42]  James F. Quinn,et al.  On Hypothesis Testing in Ecology and Evolution , 1983, The American Naturalist.

[43]  G. T. Jones ‘Surely You're Joking, Mr Feynman!’ Adventures of a Curious Character , 1985 .

[44]  Jim M Cushing,et al.  Chaos in Ecology: Experimental Nonlinear Dynamics , 2002 .

[45]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[46]  Daniel Simberloff,et al.  Competition Theory, Hypothesis-Testing, and Other Community Ecological Buzzwords , 1983, The American Naturalist.

[47]  Laurence D. Mueller,et al.  Stability in Model Populations , 2000 .

[48]  Raphael Silberzahn,et al.  Crowdsourced research: Many hands make tight work , 2015, Nature.

[49]  T. C. Chamberlin The Method of Multiple Working Hypotheses: With this method the dangers of parental affection for a favorite theory can be circumvented. , 1965, Science.

[50]  Ullica Segerstrale,et al.  Defenders Of The Truth: The Battle For Science In The Sociobiology Debate And Beyond , 2000 .

[51]  A. Burls Critical appraisal , 2016, Australasian psychiatry : bulletin of Royal Australian and New Zealand College of Psychiatrists.

[52]  M. Saunders,et al.  Plant-Provided Food for Carnivorous Insects: a Protective Mutualism and Its Applications , 2009 .

[53]  S. Ellner,et al.  Rapid evolution drives ecological dynamics in a predator–prey system , 2003, Nature.

[54]  H. Wilbur,et al.  EXPERIMENTAL ECOLOGY OF FOOD WEBS: COMPLEX SYSTEMS IN TEMPORARY PONDS , 1997 .

[55]  S. Fretwell,et al.  On territorial behavior and other factors influencing habitat distribution in birds , 1969 .

[56]  Ilkka Hanski,et al.  Contrasting alternative hypotheses about rodent cycles by translating them into parameterized models , 2001 .

[57]  G. De’ath,et al.  CLASSIFICATION AND REGRESSION TREES: A POWERFUL YET SIMPLE TECHNIQUE FOR ECOLOGICAL DATA ANALYSIS , 2000 .

[58]  Peter B. Adler,et al.  Finding generality in ecology: a model for globally distributed experiments , 2014 .

[59]  N. McGlynn Thinking fast and slow. , 2014, Australian veterinary journal.

[60]  Leif D. Nelson,et al.  False-Positive Psychology , 2011, Psychological science.

[61]  W. M. Wood-Vasey,et al.  Measurement of Ωm, ΩΛ from a Blind Analysis of Type Ia Supernovae with CMAGIC: Using Color Information to Verify the Acceleration of the Universe , 2006, astro-ph/0602411.

[62]  Nicholas J Gotelli,et al.  P values, hypothesis testing, and model selection: it's déjà vu all over again. , 2014, Ecology.

[63]  Jonathan Roughgarden,et al.  Competition and Theory in Community Ecology , 1983, The American Naturalist.