Simplicity is not Truth-Indicative

In this paper I will argue that, in general, where the evidence supports two theories equally, the simpler theory is not more likely to be true and is not likely to be nearer the truth. In other words simplicity does not tell us anything about model bias. Our preference for simpler theories (apart from their obvious pragmatic advantages) can be explained by the facts that humans are known to elaborate unsuccessful theories rather than attempt a thorough revision and that a fixed set of data can only justify adjusting a certain number of parameters to a limited degree of precision. No extra tendency towards simplicity in the natural world is necessary to explain our preference for simpler theories. Thus Occam's razor eliminates itself (when interpreted in this form).

[1]  N. Draper,et al.  Applied Regression Analysis. , 1967 .

[2]  Cullen Schaffer,et al.  A Conservation Law for Generalization Performance , 1994, ICML.

[3]  Daniel N. Osherson,et al.  On Advancing Simple Hypotheses , 1990, Philosophy of Science.

[4]  T. Kuhn,et al.  The Structure of Scientific Revolutions , 1963 .

[5]  N. Chater.  The Search for Simplicity: A Fundamental Cognitive Principle? , 1999 .

[6]  Ilya Gertsbakh,et al.  Measurement theory for engineers , 2003 .

[7]  Bruce Edmonds,et al.  From KISS to KIDS - An 'Anti-simplistic' Modelling Approach , 2004, MABS.

[8]  W. Cleveland,et al.  Regression by local fitting: Methods, properties, and computational algorithms , 1988 .

[9]  Judea Pearl,et al.  ON THE CONNECTION BETWEEN THE COMPLEXITY AND CREDIBILITY OF INFERRED MODELS , 1978 .

[10]  Geoffrey I. Webb.  Further Experimental Evidence against the Utility of Occam's Razor , 1996, J. Artif. Intell. Res..

[11]  M. Forster,et al.  Model Selection in Science: The Problem of Language Variance , 1999, The British Journal for the Philosophy of Science.

[12]  Mario Bunge,et al.  The Myth Of Simplicity , 1963 .

[13]  N. Goodman,et al.  The Structure of Appearance. , 1953 .

[14]  David M. Raup,et al.  How Nature Works: The Science of Self-Organized Criticality , 1997 .

[15]  Karl R. Popper.  The Logic of Scientific Discovery. , 1977 .

[16]  Pedro M. Domingos.  Beyond Occam's Razor: Process-Oriented Evaluation , 2000, ECML.

[17]  I. Good.  Corroboration, Explanation, Evolving Probability, Simplicity and a Sharpened Razor , 1968, The British Journal for the Philosophy of Science.

[18]  V. Vapnik,et al.  Necessary and Sufficient Conditions for the Uniform Convergence of Means to their Expectations , 1982 .

[19]  Peter D. Turney.  The Curve Fitting Problem: A Solution1 , 1990, The British Journal for the Philosophy of Science.

[20]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[21]  Ming Li,et al.  Philosophical Issues in Kolmogorov Complexity , 1992, ICALP.

[22]  Albert Einstein.  Relativity: The Special and the General Theory A Popular Exposition , 1920, Nature.

[23]  P. Murphy.  An Empirical Analysis of the Bene t of Decision Tree Size Biases as a Function of Concept Distribution , 1994 .

[24]  Gary James Jason,et al.  The Logic of Scientific Discovery , 1988 .

[25]  David Miller,et al.  Inference, method, and decision , 1977 .

[26]  Malcolm R. Forster,et al.  How to Tell When Simpler, More Unified, or Less Ad Hoc Theories will Provide More Accurate Predictions , 1994, The British Journal for the Philosophy of Science.

[27]  Russell Greiner,et al.  Exploring the Decision Forest: An Empirical Investigation of Occam's Razor in Decision Tree Induction , 1997 .

[28]  David H. Wolpert,et al.  The Lack of A Priori Distinctions Between Learning Algorithms , 1996, Neural Computation.

[29]  J. Kemeny.  Two Measures of Complexity , 1955 .

[30]  W. Quine,et al.  On simple theories of a complex world , 2004, Synthese.

[31]  S. Moss,et al.  A smart automated macroeconometric forecasting system , 1994 .