Semi-rational models of conditioning: The case of trial order

© Oxford University Press, 2008. All rights reserved. This chapter considers the question of how learning adapts to changing environments, with particular reference to animal studies of operant and classical conditioning. It discusses a variety of probabilistic models, with different assumptions concerning the environment; and contrasts this type of model with a model by Kruschke (2006) which carries out local, approximate, Bayesian inference. It further suggests that it may be too early to incorporate mechanistic limitations into models of conditioning - enriching the understanding of the environment, and working with a 'pure' Bayesian rational analysis for that environment, may provide an alternative, and perhaps theoretically more elegant, way forward.

[1]  H. Simon,et al.  A Behavioral Model of Rational Choice , 1955 .

[2]  E. W. Adams,et al.  Models of Man, Social and Rational: Mathematical Essays on Rational Human Behavior in a Social Setting , 1962 .

[3]  Noam Chomsky,et al.  वाक्यविन्यास का सैद्धान्तिक पक्ष = Aspects of the theory of syntax , 1965 .

[4]  L. Kamin.  Predictability, surprise, attention, and conditioning , 1967 .

[5]  J. Konorski.  Integrative activity of the brain , 1967 .

[6]  B. Campbell,et al.  Punishment and aversive behavior , 1969 .

[7]  R. Rescorla.  A theory of pavlovian conditioning: The effectiveness of reinforcement and non-reinforcement , 1972 .

[8]  N. Mackintosh.  A Theory of Attention: Variations in the Associability of Stimuli with Reinforcement , 1975 .

[9]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[10]  D. Shanks.  Forward and Backward Blocking in Human Contingency Judgement , 1985 .

[11]  G. Gillett.  Neurophilosophy: Toward a Unified Science of the Mind-Brain , 1987 .

[12]  D. Medin,et al.  Problem structure and the use of base-rate information from experience. , 1988, Journal of experimental psychology. General.

[13]  Lynn Waterhouse,et al.  Neurophilosophy: Toward a Unified Science of the Mind/Brain , 1988 .

[14]  D. Medin,et al.  Sensitivity to changes in base-rate information , 1991 .

[15]  Michael I. Jordan,et al.  Task Decomposition Through Competition in a Modular Connectionist Architecture: The What and Where Vision Tasks , 1990, Cogn. Sci..

[16]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[17]  Geoffrey E. Hinton,et al.  Keeping the neural networks simple by minimizing the description length of the weights , 1993, COLT '93.

[18]  Steve R. Waterhouse,et al.  Bayesian Methods for Mixtures of Experts , 1995, NIPS.

[19]  R. R. Miller,et al.  Biological significance in forward and backward blocking: resolution of a discrepancy between animal conditioning and human causal judgment. , 1996, Journal of experimental psychology. General.

[20]  Peter Dayan,et al.  Statistical Models of Conditioning , 1997, NIPS.

[21]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[22]  E. Wasserman,et al.  Backward Blocking and Recovery from Overshadowing in Human Causal Judgement: The Role of Within-compound Associations , 1998, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[23]  Stuart J. Russell,et al.  Bayesian Q-Learning , 1998, AAAI/IAAI.

[24]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.

[25]  Geoffrey E. Hinton.  Products of experts , 1999 .

[26]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[27]  S. Kakade,et al.  Learning and selective attention , 2000, Nature Neuroscience.

[28]  Peter Dayan,et al.  Explaining Away in Weight Space , 2000, NIPS.

[29]  J. Kruschke.  Toward a unified model of attention in associative learning , 2001 .

[30]  Tom Minka,et al.  A family of algorithms for approximate Bayesian inference , 2001 .

[31]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[32]  Peter Dayan,et al.  Expected and Unexpected Uncertainty: ACh and NE in the Neocortex , 2002, NIPS.

[33]  H. Pashler.  STEVENS' HANDBOOK OF EXPERIMENTAL PSYCHOLOGY , 2002 .

[34]  S. Kakade,et al.  Acquisition and extinction in autoshaping. , 2002, Psychological review.

[35]  David S. Touretzky,et al.  Model Uncertainty in Classical Conditioning , 2003, NIPS.

[36]  J. Kruschke.  Attention in Learning , 2003 .

[37]  Christopher J. Mitchell,et al.  Forward and backward blocking of causal judgment is enhanced by additivity of effect magnitude , 2003, Memory & cognition.

[38]  David S. Touretzky,et al.  Similarity and Discrimination in Classical Conditioning: A Latent Variable Account , 2004, NIPS.

[39]  Henrik Madsen,et al.  On the construction of a reduced rank square-root Kalman filter for efficient uncertainty propagation , 2005, Future Gener. Comput. Syst..

[40]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[41]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[42]  B. Balleine,et al.  Parallel incentive processing: an integrated view of amygdala function , 2006, Trends in Neurosciences.

[43]  J. Kruschke.  Locally Bayesian learning with applications to retrospective revaluation and highlighting. , 2006, Psychological review.

[44]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[45]  Aaron C. Courville,et al.  Bayesian theories of conditioning in a changing world , 2006, Trends in Cognitive Sciences.

[46]  Thomas L. Griffiths,et al.  Technical Introduction: A primer on probabilistic inference , 2005 .

[47]  Michael R. Waldmann,et al.  Causal learning in rats and humans: A minimal rational model , 2008 .