Evidence, Evidence Functions, and Error Probabilities

Publisher Summary This chapter presents statistics as a tool for the development of scientific knowledge. It helps to develop the desired data for knowledge developing tools in science and shows how far the evidential statistical paradigm goes toward meeting these objectives. It also relates evidential statistics to the competing paradigms of Bayesian statistics and error statistics. Individual scientists have personal beliefs regarding the theories and even the observations of science. The statistical evidence measures the differences of models from truth in a single dimension and consequently may flatten some of the richness of a linguistic theory. The issue of quantifying evidence in the data has always vexed statisticians. Under the evidential paradigm, the composite hypothesis problem can be recast as a model selection problem among models with different numbers of free parameters. The elevation of evidentialism to a practical statistical approach is due to Royall's introduction of the concepts of weak and strong evidence and of misleading evidence.

[1]  Brian Dennis,et al.  Data cloning: easy maximum likelihood estimation for complex ecological models using Bayesian Markov chain Monte Carlo methods. , 2007, Ecology letters.

[2]  Malcolm R. Forster,et al.  Counterexamples to a likelihood theory of evidence , 2006, Minds and Machines.

[3]  Jeffrey E. Jarrett.  The Nature of Statistical Evidence , 2008, Technometrics.

[4]  Deborah G. Mayo,et al.  Error and the Growth of Experimental Knowledge , 1996 .

[5]  David Miller.  The truth-likeness of truthlikeness , 1972 .

[6]  R. Fisher.  014: On the "Probable Error" of a Coefficient of Correlation Deduced from a Small Sample. , 1921 .

[7]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[8]  R. Royall.  Statistical Evidence: A Likelihood Paradigm , 1997 .

[9]  Yuehua Wu,et al.  Model selection with data-oriented penalty , 1999 .

[10]  R. Royall,et al.  The Likelihood Paradigm for Statistical Evidence , 2004 .

[11]  Brian Dennis,et al.  Hierarchical models in ecology: confidence intervals, hypothesis testing, and model selection using data cloning. , 2009, Ecology.

[12]  P. McCullagh,et al.  Generalized Linear Models, 2nd Edn. , 1990 .

[13]  K. Popper,et al.  Logik der Forschung , 1935 .

[14]  Robert A. Nisbet,et al.  History of the Idea of Progress , 1980 .

[15]  G. Casella,et al.  Statistical Inference , 2003, Encyclopedia of Social Network Analysis and Mining.

[16]  David Miller,et al.  Sokal & Bricmont: Back to the Frying Pan * , 2000 .

[17]  Mary S. Morgan,et al.  Learning from Models , 1997 .

[18]  Leonard J. Savage,et al.  On Rereading R. A. Fisher , 1976 .

[19]  Anthony C. Davison,et al.  Bootstrap Methods and Their Application , 1998 .

[20]  Nancy Cartwright,et al.  The Dappled World: Introduction , 1999 .

[21]  D L DeMets,et al.  Interim analysis: the alpha spending function approach. , 1994, Statistics in medicine.

[22]  Bas C. van Fraassen,et al.  The Scientific Image , 1980 .

[23]  John H. Harris.  POPPER'S DEFINITIONS OF ‘VERISIMILITUDE’1 , 1974, The British Journal for the Philosophy of Science.

[24]  Seymour Geisser,et al.  Logic of Statistical Inference. , 1967 .

[25]  R. Royall.  On the Probability of Observing Misleading Statistical Evidence , 2000 .

[26]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[27]  S. Lele,et al.  The nature of scientific evidence : statistical, philosophical, and empirical considerations , 2004 .

[28]  R. Wolpert,et al.  Likelihood Principle , 2022, The SAGE Encyclopedia of Research Design.

[29]  David W. Miller.  POPPER'S QUALITATIVE THEORY OF VERISIMILITUDE , 1974, The British Journal for the Philosophy of Science.

[30]  Karl R. Popper.  A NOTE ON VERISMILITUDE , 1976, The British Journal for the Philosophy of Science.

[31]  David Miller.  ON THE COMPARISON OF FALSE THEORIES BY THEIR BASES , 1974, The British Journal for the Philosophy of Science.

[32]  Bruce G. Lindsay.  Statistical Distances as Loss Functions in Assessing Model Adequacy , 2004 .

[33]  Lisa J Strug,et al.  An Alternative Foundation for the Planning and Evaluation of Linkage Analysis , 2006, Human Heredity.

[34]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[35]  David R. Anderson,et al.  Avoiding pitfalls when using information-theoretic methods , 2002 .

[36]  Marianthi Markatou,et al.  Quadratic distances on probabilities: A unified foundation , 2008, 0804.0991.

[37]  Van Fraassen,et al.  The Empirical Stance , 2002 .

[38]  M. J. Bayarri,et al.  Calibration of ρ Values for Testing Precise Null Hypotheses , 2001 .

[39]  Gary James Jason,et al.  The Logic of Scientific Discovery , 1988 .

[40]  P. McCullagh,et al.  Generalized Linear Models , 1984 .

[41]  Donald Fraser,et al.  On the Sufficiency and Likelihood Principles , 1963 .

[42]  S. Goodman.  A dirty dozen: twelve p-value misconceptions. , 2008, Seminars in hematology.

[43]  Zucchini,et al.  An Introduction to Model Selection. , 2000, Journal of mathematical psychology.

[44]  J. Platt.  Strong Inference: Certain systematic methods of scientific thinking may produce much more rapid progress than others. , 1964, Science.

[45]  K. Popper,et al.  Conjectures and refutations;: The growth of scientific knowledge , 1972 .

[46]  Mark L. Taper,et al.  The northern Yellowstone elk: density dependence and climatic conditions , 2002 .

[47]  David R. Anderson,et al.  Null Hypothesis Testing: Problems, Prevalence, and an Alternative , 2000 .

[48]  J. Tukey.  Conclusions vs Decisions , 1960 .

[49]  Willard Van OrmanQuine,et al.  Two Dogmas of Empiricism 1 a , 2008 .

[50]  R. Hughes.  Models and Representation , 1997, Philosophy of Science.

[51]  Jeffrey F Peipert,et al.  What your statistician never told you about P-values. , 2003, The Journal of the American Association of Gynecologic Laparoscopists.

[52]  P. Tichý,et al.  ON POPPER'S DEFINITIONS OF VERISIMILITUDE1 , 1974, The British Journal for the Philosophy of Science.

[53]  R. Giere.  How Models Are Used to Represent Reality , 2004, Philosophy of Science.

[54]  Donald A Berry,et al.  The difficult and ubiquitous problems of multiplicities , 2007, Pharmaceutical statistics.

[55]  Subhash R. Lele,et al.  Evidence Functions and the Optimality of the Law of Likelihood , 2004 .

[56]  N. Reid,et al.  Likelihood , 1993 .

[57]  David Miller,et al.  Out of Error: Further Essays on Critical Rationalism , 2006 .

[58]  N. Cartwright.  Book review: introduction and reply to - the dappled world: a study of the boundaries of science , 2002 .

[59]  Jeffrey D Blume,et al.  Likelihood methods for measuring statistical evidence , 2002, Statistics in medicine.

[60]  J. Platt.  Strong Inference , 2007 .

[61]  Deborah G. Mayo.  An Error-Statistical Philosophy of Evidence , 2004 .

[62]  Roman Frigg,et al.  Scientific Representation and the Semantic View of Theories , 2006, THEORIA.

[63]  D. Cox,et al.  Frequentist statistics as a theory of inductive inference , 2006, math/0610846.

[64]  Malcolm R. Forster,et al.  Predictive Accuracy as an Achievable Goal of Science , 2002, Philosophy of Science.

[65]  Jeffrey D. Blume,et al.  How Often Likelihood Ratios are Misleading in Sequential Trials , 2008 .

[66]  Walter Zucchini,et al.  Model Selection , 2011, International Encyclopedia of Statistical Science.

[67]  I. Niiniluoto.  Verisimilitude: The Third Period , 1998, The British Journal for the Philosophy of Science.

[68]  R. Fisher,et al.  On the Mathematical Foundations of Theoretical Statistics , 1922 .

[69]  Ken Gemes,et al.  Verisimilitude and Content , 2005, Synthese.

[70]  R. Royall.  The Effect of Sample Size on the Meaning of Significance Tests , 1986 .

[71]  D. Mayo,et al.  Severe Testing as a Basic Concept in a Neyman–Pearson Philosophy of Induction , 2006, The British Journal for the Philosophy of Science.

[72]  W. Quine.  Main trends in recent philosophy: two dogmas of empiricism. , 1951 .

[73]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[74]  A. J. Willis.  Ecological understanding. The nature of theory and the theory of nature , 1995 .

[75]  G. Box.  Robustness in the Strategy of Scientific Model Building. , 1979 .

[76]  R. Fisher.  001: On an Absolute Criterion for Fitting Frequency Curves. , 1912 .

[77]  Mark L. Taper,et al.  Model Identification from Many Candidates , 2004 .

[78]  A. Goldman.  Epistemology and Cognition , 1986 .

[79]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .