Inductive Learning in Small and Large Worlds

Bayesian treatments of inductive inference and decision making presuppose that the structure of the situation under consideration is fully known. We are, however, often faced with having only very fragmentary information about an epistemic situation. This tension was discussed in decision theory by Savage (1954) in terms of ‘small worlds’ and ‘large worlds’ (‘grand worlds’ in Savage’s terminology). Large worlds allow a more fine grained description of a situation than small worlds. The additional distinctions may not be readily accessible, though. Savage realized that planning ahead is difficult, if not impossible, whenever we don’t have sufficient knowledge of the the basic structure of an epistemic situation. Since planning is at the heart of Bayesian inductive inference and decision making, it is not easy to see how a Bayesian—or any learner, for that matter—can deal with incomplete information. The aim of this paper is to outline how the mathematical and philosophical foundations of inductive learning in large worlds may be developed. First I wish to show that there is an important sense in which Bayesian solutions for inductive learning in large worlds exist. The basic idea is the following: Even if one’s knowledge of a situation is incomplete and restricted, Bayesian methods can be applied based on the information that is available. This idea is more fully developed in §§4, 5 and 6 for two concrete inductive learning rules that I introduce in §2. Importantly, however, this does not always lead to fully rational inductive learning: the analysis of a learning rule within the confines of the available information is by itself not sufficient to establish its rationality in larger contexts. In §3 I couch this problem in terms of Savage’s discussion of small worlds and large worlds. I take this thread up again in §7, where inductive learning in small and large worlds is examined in the light of bounded rationality and Richard Jeffrey’s epistemology of ‘radical probabilism’.

[1]  Hans Reichenbach,et al.  The theory of probability , 1968 .

[2]  David A. Freedman,et al.  De Finetti's generalizations of exchangeability , 1980 .

[3]  Gerd Gigerenzer,et al.  Are rational actor models "rational" outside small worlds? , 2012 .

[4]  Gerd Gigerenzer,et al.  Heuristic decision making. , 2011, Annual review of psychology.

[5]  Brian Skyrms,et al.  Carnapian inductive logic and Bayesian statistics , 1996 .

[6]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[7]  Dan W. Brockt,et al.  The Theory of Justice , 2017 .

[8]  P. W. Jones,et al.  Bandit Problems, Sequential Allocation of Experiments , 1987 .

[9]  C. Howson.  Hume's Problem , 2000 .

[10]  H. Simon,et al.  Models of Man. , 1957 .

[11]  W. E. Johnson.  Logic - Part III - The Logical Foundations of Science (1924) , 1927 .

[12]  B. Skyrms.  Carnapian Inductive Logic for a Value Continuum , 1993 .

[13]  N. Goodman.  Fact, Fiction, and Forecast , 1955 .

[14]  Sanjeev R. Kulkarni,et al.  Reliable Reasoning: Induction and Statistical Learning Theory , 2007 .

[15]  Colin Camerer,et al.  Experience‐weighted Attraction Learning in Normal Form Games , 1999 .

[16]  Richard Bradley,et al.  Radical Probabilism and Bayesian Conditioning* , 2005, Philosophy of Science.

[17]  H. Simon,et al.  A Behavioral Model of Rational Choice , 1955 .

[18]  W. E. Johnson.  I.—PROBABILITY: THE DEDUCTIVE AND INDUCTIVE PROBLEMS , 1932 .

[19]  Domenico Costantani.  The relevance quotient , 1979 .

[20]  D. Freedman.  Mixtures of Markov Processes , 1962 .

[21]  D. Aldous.  Exchangeability and related topics , 1985 .

[22]  H. Peyton Young,et al.  Strategic Learning and Its Limits , 2004 .

[23]  Simon M. Huttegger.  Signals: Evolution, Learning and InformationBy Brian Skyrms , 2011 .

[24]  Gilbert Harman.  Reflections on Knowledge and its Limits , 2002 .

[25]  R. Jeffrey.  Probability and the Art of Judgment , 1992 .

[26]  James M. Joyce.  The Foundations of Causal Decision Theory , 1999 .

[27]  B. Finetti.  La probabilitaà e la statistica nei rapporti con l'Induzione, Secondo i diversi Punti di vista , 2011 .

[28]  C. Aitken,et al.  The logic of decision , 2014 .

[29]  S. Zabell.  W. E. Johnson's "Sufficientness" Postulate , 1982 .

[30]  S. Zabell.  Characterizing Markov exchangeable sequences , 1995 .

[31]  Sandy L. Zabell,et al.  The rule of succession , 1989 .

[32]  H. Jeffreys,et al.  Theory of probability , 1896 .

[33]  James M. Joyce.  The Foundations of Causal Decision Theory by James M. Joyce , 1999 .

[34]  J. Romeijn.  Analogical Predictions for Explicit Similarity , 2006 .

[35]  J. Schreiber.  Foundations Of Statistics , 2016 .

[36]  B. D. Finetti.  La prévision : ses lois logiques, ses sources subjectives , 1937 .

[37]  Leonard J. Savage,et al.  Implications of Personal Probability for Induction , 1967 .

[38]  David Lindley,et al.  Logical foundations of probability , 1951 .

[39]  J. Neumann,et al.  Theory of games and economic behavior , 1945, 100 Years of Math Milestones.