How to Tell When Simpler, More Unified, or Less Ad Hoc Theories will Provide More Accurate Predictions

Traditional analyses of the curve fitting problem maintain that the data do not indicate what form the fitted curve should take. Rather, this issue is said to be settled by prior probabilities, by simplicity, or by a background theory. In this paper, we describe a result due to Akaike [1973], which shows how the data can underwrite an inference concerning the curve's form based on an estimate of how predictively accurate it will be. We argue that this approach throws light on the theoretical virtues of parsimoniousness, unification, and non ad hocness, on the dispute about Bayesianism, and on empiricism and scientific realism.

[1]  N. Cartwright.  How the laws of physics lie , 1984 .

[2]  A. Garrett,et al.  Ockham’s Razor , 1991 .

[3]  H. Kyburg,et al.  How the laws of physics lie , 1984 .

[4]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[5]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[6]  David W. Miller.  POPPER'S QUALITATIVE THEORY OF VERISIMILITUDE , 1974, The British Journal for the Philosophy of Science.

[7]  J. Ware,et al.  Applications of Statistics , 1978 .

[8]  D. Lindley.  A STATISTICAL PARADOX , 1957 .

[9]  Ellery Eells,et al.  PROBLEMS OF OLD EVIDENCE , 1985 .

[10]  Malcolm R. Forster,et al.  UNIFICATION, EXPLANATION, AND THE COMPOSITION OF CAUSES IN NEWTONIAN MECHANICS , 1988 .

[11]  Malcolm R. Forster,et al.  Non-bayesian foundations for statistical estimation, prediction, and the ravens example , 1994 .

[12]  William Harper.  Consilience and Natural Kind Reasoning , 1989 .

[13]  R. Kapp.  British Journal for the Philosophy of Science , 1950, Nature.

[14]  I. Lakatos.  Falsification and the Methodology of Scientific Research Programmes , 1976 .

[15]  P. Churchland.  A Neurocomputational Perspective: The Nature of Mind and the Structure of Science , 1989 .

[16]  N. Swerdlow,et al.  Copernican revolution , 1975, Nature.

[17]  A. Musgrave,et al.  Logical versus Historical Theories of Confirmation , 1974, The British Journal for the Philosophy of Science.

[18]  Elliott Sober,et al.  Explanation and its Limits: Let's Razor Ockham's Razor , 1991 .

[19]  L. Laudan.  A Confutation of Convergent Realism , 1981, Philosophy of Science.

[20]  G. Kitagawa,et al.  Akaike Information Criterion Statistics , 1988 .

[21]  M. Aitkin.  Posterior Bayes Factors , 1991 .

[22]  Richard Healey,et al.  Foundations of Space-Time Theories , 1983 .

[23]  石黒 真木夫,et al.  Akaike information criterion statistics , 1986 .

[24]  P. Tichý,et al.  ON POPPER'S DEFINITIONS OF VERISIMILITUDE1 , 1974, The British Journal for the Philosophy of Science.

[25]  H. Akaike.  Prediction and Entropy , 1985 .

[26]  John E. Moody,et al.  The Effective Number of Parameters: An Analysis of Generalization and Regularization in Nonlinear Learning Systems , 1991, NIPS.

[27]  J. Farris.  CONJECTURES AND REFUTATIONS , 1995, Cladistics : the international journal of the Willi Hennig Society.

[28]  Peter Godfrey-Smith,et al.  Reconstructing the Past: Parsimony, Evolution, and Inference , 1989 .

[29]  Peter D. Turney.  The Curve Fitting Problem: A Solution1 , 1990, The British Journal for the Philosophy of Science.

[30]  Gary James Jason,et al.  The Logic of Scientific Discovery , 1988 .

[31]  William Whewell,et al.  The philosophy of the inductive sciences , 1847 .

[32]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[33]  H. Akaike.  A new look at the statistical model identification , 1974 .

[34]  David Miller,et al.  Inference, method, and decision , 1977 .

[35]  T. Broadbent,et al.  Criticism and the Growth of Knowledge , 1972 .

[36]  James O. Berger,et al.  Ockham's Razor and Bayesian Analysis , 1992 .

[37]  Elliott Sober,et al.  Likelihood and Convergence , 1988, Philosophy of Science.

[38]  Malcolm R. Forster.  Unification and Scientific Realism Revisited , 1986, PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association.

[39]  Peter Urbach,et al.  Scientific Reasoning: The Bayesian Approach , 1989 .

[40]  M. Kendall,et al.  The Logic of Scientific Discovery. , 1959 .

[41]  Hirotugu Akaike,et al.  On entropy maximization principle , 1977 .

[42]  Gareth Nelson,et al.  Reconstructing the Past: Parsimony, Evolution, and Inference , 1989 .

[43]  Graham Priest.  Gruesome Simplicity , 1976, Philosophy of Science.

[44]  J. Woodward,et al.  Saving the phenomena , 1988 .

[45]  Bas C. van Fraassen,et al.  The Scientific Image , 1980 .

[46]  Stephen E. Fienberg,et al.  A Celebration of Statistics , 1985 .

[47]  Shun-ichi Amari,et al.  Network information criterion-determining the number of hidden units for an artificial neural network model , 1994, IEEE Trans. Neural Networks.

[48]  Cliff Hooker,et al.  A Realistic Theory of Science. , 1987 .

[49]  D. Knowles,et al.  Explanation and its Limits , 1991 .