A modern method of multiple working hypotheses to improve inference in ecology

Science provides a method to learn about the relationships between observed patterns and the processes that generate them. However, inference can be confounded when an observed pattern cannot be clearly and wholly attributed to a hypothesized process. Over-reliance on traditional single-hypothesis methods (i.e. null hypothesis significance testing) has resulted in replication crises in several disciplines, and ecology exhibits features common to these fields (e.g. low-power study designs, questionable research practices, etc.). Considering multiple working hypotheses in combination with pre-data collection modelling can be an effective means to mitigate many of these problems. We present a framework for explicitly modelling systems in which relevant processes are commonly omitted, overlooked or not considered and provide a formal workflow for a pre-data collection analysis of multiple candidate hypotheses. We advocate for and suggest ways that pre-data collection modelling can be combined with consideration of multiple working hypotheses to improve the efficiency and accuracy of research in ecology.

[1]  Sander Greenland,et al.  Scientists rise up against statistical significance , 2019, Nature.

[2]  James F. Quinn,et al.  On Hypothesis Testing in Ecology and Evolution , 1983, The American Naturalist.

[3]  T. C. CHAMBERLIN The Method of Multiple Working Hypotheses , 1931, The Journal of Geology.

[4]  David R. Anderson,et al.  Model Selection and Multimodel Inference , 2003 .

[5]  G. E. Hutchinson,et al.  The Balance of Nature and Human Impact: The paradox of the plankton , 2013 .

[6]  A. Greenwald Consequences of Prejudice Against the Null Hypothesis , 1975 .

[7]  M. Kondoh Unifying the relationships of species richness to productivity and disturbance , 2001, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[8]  J. Platt Strong Inference , 2007 .

[9]  T. C. Chamberlin The Method of Multiple Working Hypotheses , 1931, The Journal of Geology.

[10]  L. HARKing: Hypothesizing After the Results are Known , 2002 .

[11]  C. McCain,et al.  Natural population variability may be masking the more-individuals hypothesis. , 2020, Ecology.

[12]  C. Glenn Begley,et al.  Raise standards for preclinical cancer research , 2012 .

[13]  Fintan R. Steele,et al.  The Demon-Haunted World: Science as a Candle in the Dark , 1996, Nature Medicine.

[14]  C. Begley,et al.  Drug development: Raise standards for preclinical cancer research , 2012, Nature.

[15]  J. Fox,et al.  The intermediate disturbance hypothesis should be abandoned. , 2013, Trends in ecology & evolution.

[16]  A. C. James,et al.  On the Use of Stable Isotopes in Trophic Ecology , 2011 .

[17]  W. Hillis Why physicists like models and why biologists should , 1993, Current Biology.

[18]  Brian J. McGill,et al.  Can niche-based distribution models outperform spatial interpolation? , 2007 .

[19]  J. Andrew Royle,et al.  ESTIMATING SITE OCCUPANCY RATES WHEN DETECTION PROBABILITIES ARE LESS THAN ONE , 2002, Ecology.

[20]  Jacob Cohen The earth is round (p < .05) , 1994 .

[21]  Carl T. Bergstrom,et al.  Publication bias and the canonization of false facts , 2016, eLife.

[22]  P. Chesson Mechanisms of Maintenance of Species Diversity , 2000 .

[23]  E. Revilla,et al.  A movement ecology paradigm for unifying organismal movement research , 2008, Proceedings of the National Academy of Sciences.

[24]  T. Johansson Hail the impossible: p-values, evidence, and likelihood. , 2011, Scandinavian journal of psychology.

[25]  Peter B Adler,et al.  A niche for neutrality. , 2007, Ecology letters.

[26]  David Colquhoun,et al.  The reproducibility of research and the misinterpretation of p-values , 2017, bioRxiv.

[27]  David R. Anderson,et al.  AIC model selection and multimodel inference in behavioral ecology: some background, observations, and comparisons , 2011, Behavioral Ecology and Sociobiology.

[28]  Timothy L Lash,et al.  The Harm Done to Reproducibility by the Culture of Null Hypothesis Significance Testing. , 2017, American journal of epidemiology.

[29]  Gary James Jason,et al.  The Logic of Scientific Discovery , 1988 .

[30]  Andreas Huth,et al.  Statistical inference for stochastic simulation models--theory and application. , 2011, Ecology letters.

[31]  R. Freckleton,et al.  The Ecological Detective: Confronting Models with Data , 1999 .

[32]  T. Rothenberg Identification in Parametric Models , 1971 .

[33]  Brian A. Nosek,et al.  Power failure: why small sample size undermines the reliability of neuroscience , 2013, Nature Reviews Neuroscience.

[34]  Philip A. Stephens,et al.  Inference in ecology and evolution. , 2007, Trends in ecology & evolution.

[35]  R. Greenberg,et al.  An experimental study of habitat selection by birds in a coffee plantation. , 2008, Ecology.

[36]  J. Haldane A Defense of Beanbag Genetics , 2015, Perspectives in biology and medicine.

[37]  Andrew Gelman,et al.  Measurement error and the replication crisis , 2017, Science.

[38]  D. Simberloff A succession of paradigms in ecology: Essentialism to materialism and probabilism , 2004, Synthese.

[39]  Michael C. Frank,et al.  Estimating the reproducibility of psychological science , 2015, Science.

[40]  Robert L. Wolpert,et al.  Statistical Inference , 2019, Encyclopedia of Social Network Analysis and Mining.

[41]  D. Bakan,et al.  The test of significance in psychological research. , 1966, Psychological bulletin.

[42]  J. Ioannidis Why Most Published Research Findings Are False , 2005, PLoS medicine.

[43]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[44]  Dominique G. Roche,et al.  Ocean acidification does not impair the behaviour of coral reef fishes , 2020, Nature.

[45]  N. Lazar,et al.  The ASA Statement on p-Values: Context, Process, and Purpose , 2016 .

[46]  T. Koopmans,et al.  The Identification of Structural Characteristics , 1950 .

[47]  James D. Nichols,et al.  Accumulating evidence in ecology: Once is not enough , 2019, Ecology and evolution.

[48]  Maureen A. O’Malley,et al.  Do simple models lead to generality in ecology? , 2013, Trends in ecology & evolution.

[49]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[50]  David R. Anderson,et al.  Kullback-Leibler information as a basis for strong inference in ecological studies , 2001 .

[51]  D. Fanelli How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data , 2009, PloS one.

[52]  John P. A. Ioannidis,et al.  p-Curve and p-Hacking in Observational Research , 2016, PloS one.

[53]  J. Platt Strong Inference: Certain systematic methods of scientific thinking may produce much more rapid progress than others. , 1964, Science.

[54]  S. Goodman,et al.  Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations , 2016, European Journal of Epidemiology.

[55]  F. Fidler,et al.  Questionable research practices in ecology and evolution , 2018, PloS one.

[56]  C. S. Holling CHAPTER 8 – The Strategy of Building Models of Complex Ecological Systems , 1966 .

[57]  Yoan Fourcade,et al.  Paintings predict the distribution of species, or the challenge of selecting environmental predictors and evaluation statistics , 2018 .

[58]  D. Burslem,et al.  Defining and defending Connell's intermediate disturbance hypothesis: a response to Fox. , 2013, Trends in ecology & evolution.

[59]  R. Lewontin,et al.  Holism and reductionism in ecology , 1994 .

[60]  J. Connell Diversity in tropical rain forests and coral reefs. , 1978, Science.

[61]  R. Kays,et al.  Terrestrial animal tracking as an eye on life and planet , 2015, Science.

[62]  David R. Anderson,et al.  Practical Use of the Information-Theoretic Approach , 1998 .

[63]  Caitlin A Stern,et al.  Not Just a Theory—The Utility of Mathematical Models in Evolutionary Biology , 2014, PLoS biology.

[64]  Nicholas J Gotelli,et al.  P values, hypothesis testing, and model selection: it's déjà vu all over again. , 2014, Ecology.

[65]  Tal Avgar,et al.  Why are we not evaluating multiple competing hypotheses in ecology and evolution? , 2017, Royal Society Open Science.

[66]  B. Brook,et al.  Revisiting Chamberlin: Multiple Working Hypotheses for the 21st Century , 2007 .

[67]  M. Ward,et al.  Consequences of using Conspecific Attraction in Avian Conservation: A Case Study of Endangered Colonial Waterbirds , 2011 .

[68]  Richard Levins,et al.  Dialectics and reductionism in ecology , 2004, Synthese.

[69]  Graham Bell,et al.  The Distribution of Abundance in Neutral Communities , 2000, The American Naturalist.

[70]  David R. Anderson,et al.  Null Hypothesis Testing: Problems, Prevalence, and an Alternative , 2000 .

[71]  H. Caswell THEORY AND MODELS IN ECOLOGY: A DIFFERENT PERSPECTIVE , 1988 .

[72]  James S. Clark,et al.  A future for models and data in environmental science. , 2006, Trends in ecology & evolution.