Model structure adequacy analysis: selecting models on the basis of their ability to answer scientific questions

Models carry the meaning of science. This puts a tremendous burden on the process of model selection. In general practice, models are selected on the basis of their relative goodness of fit to data penalized by model complexity. However, this may not be the most effective approach for selecting models to answer a specific scientific question because model fit is sensitive to all aspects of a model, not just those relevant to the question. Model Structural Adequacy analysis is proposed as a means to select models based on their ability to answer specific scientific questions given the current understanding of the relevant aspects of the real world.

[1]  H. Akaike.  INFORMATION THEORY AS AN EXTENSION OF THE MAXIMUM LIKELIHOOD , 1973 .

[2]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[3]  H. Bozdogan.  Model selection and Akaike's Information Criterion (AIC): The general theory and its analytical extensions , 1987 .

[4]  H. Akaike.  Factor analysis and AIC , 1987 .

[5]  Brian Dennis,et al.  Estimation of Growth and Extinction Parameters for Endangered Species , 1991 .

[6]  Alan A. Berryman,et al.  Population Theory: An Essential Ingredient in Pest Prediction, Management, and Policy-making , 1991 .

[7]  W. Covington,et al.  Sustainable ecological systems: Implementing an ecological approach to land management, implications for ponderosa pine/bunchgrass ecological systems , 1993 .

[8]  Brian Dennis,et al.  DENSITY DEPENDENCE IN TIME SERIES OBSERVATIONS OF NATURAL POPULATIONS: ESTIMATION AND TESTING' , 1994 .

[9]  J. Logan,et al.  In Defense of Big Ugly Models , 1994 .

[10]  C. Chatfield.  Model uncertainty, data mining and statistical inference , 1995 .

[11]  Mark M Hooten,et al.  Distinguishing forms of statistical density dependence and independence in animal time series data using information criteria , 1995 .

[12]  Deborah G. Mayo,et al.  Error and the Growth of Experimental Knowledge , 1996 .

[13]  R. Hilborn,et al.  The Ecological Detective: Confronting Models with Data , 1997 .

[14]  Danny C. Lee,et al.  Distribution, Status, and Likely Future Trends of Bull Trout within the Columbia River and Klamath River Basins , 1997 .

[15]  Brian Dennis,et al.  COMPLEX POPULATION DYNAMICS IN THE REAL WORLD: MODELING THE INFLUENCE OF TIME-VARYING PARAMETERS AND TIME LAGS , 1998 .

[16]  T. Weaver,et al.  FLATHEAD LAKE AND RIVER SYSTEM FISHERIES STATUS REPORT , 1999 .

[17]  E. David Ford,et al.  MULTI-CRITERIA ASSESSMENT OF ECOLOGICAL PROCESS MODELS , 1999 .

[18]  Michael J. Wisdom,et al.  Life Stage Simulation Analysis: Estimating Vital-Rate Effects on Population Growth for Conservation , 2000 .

[19]  H. Caswell.  Matrix population models : construction, analysis, and interpretation , 2001 .

[20]  Mark L. Taper,et al.  The northern Yellowstone elk: density dependence and climatic conditions , 2002 .

[21]  Alan Hastings,et al.  FITTING POPULATION MODELS INCORPORATING PROCESS NOISE AND OBSERVATION ERROR , 2002 .

[22]  Zaid Abdo,et al.  Performance-based selection of likelihood models for phylogeny estimation. , 2003, Systematic biology.

[23]  Volker Grimm,et al.  Using pattern-oriented modeling for revealing hidden information: a key for reconciling ecological theory and application , 2003 .

[24]  R. Giere.  How Models Are Used to Represent Reality , 2004, Philosophy of Science.

[25]  Amitrajeet A. Batabyal.  Modeling in Natural Resource Management: Development, Interpretation, and Application , 2004 .

[26]  Brian Dennis,et al.  ESTIMATING POPULATION TREND AND PROCESS VARIATION FOR PVA IN THE PRESENCE OF SAMPLING ERROR , 2004 .

[27]  Bruce G. Lindsay.  Statistical Distances as Loss Functions in Assessing Model Adequacy , 2004 .

[28]  C. Walters,et al.  Quantitative fisheries stock assessment: Choice, dynamics and uncertainty , 2004, Reviews in Fish Biology and Fisheries.

[29]  Subhash R. Lele,et al.  Evidence Functions and the Optimality of the Law of Likelihood , 2004 .

[30]  Subhash R. Lele,et al.  Dynamical Models as Paths to Evidence in Ecology , 2004 .

[31]  Deborah G. Mayo.  An Error-Statistical Philosophy of Evidence , 2004 .

[32]  S. Lele,et al.  The nature of scientific evidence : statistical, philosophical, and empirical considerations , 2004 .

[33]  Jerald B. Johnson,et al.  Model selection in ecology and evolution. , 2004, Trends in ecology & evolution.

[34]  Mark L. Taper,et al.  Model Identification from Many Candidates , 2004 .

[35]  Mark L. Taper,et al.  Risk‐Based Viable Population Monitoring , 2005 .

[36]  Philip A. Stephens,et al.  Information theory and hypothesis testing: a call for pluralism , 2005 .

[37]  S. Lele,et al.  ESTIMATING DENSITY DEPENDENCE, PROCESS NOISE, AND OBSERVATION ERROR , 2006 .

[38]  Mark L. Taper,et al.  Observer Error Structure in Bull Trout Redd Counts in Montana Streams: Implications for Inference on True Redd Numbers , 2006 .

[39]  N Thompson Hobbs,et al.  Alternatives to statistical hypothesis testing in ecology: a guide to self teaching. , 2006, Ecological applications : a publication of the Ecological Society of America.

[40]  D. Staples,et al.  Viable population monitoring : risk-based population monitoring for threatened and endangered species with application to bull trout, Salvelinus confluentus , 2006 .