SAwSu: An Integrated Model of Associative and Reinforcement Learning

Successfully explaining and replicating the complexity and generality of human and animal learning will require the integration of a variety of learning mechanisms. Here, we introduce a computational model which integrates associative learning (AL) and reinforcement learning (RL). We contrast the integrated model with standalone AL and RL models in three simulation studies. First, a synthetic grid-navigation task is employed to highlight performance advantages for the integrated model in an environment where the reward structure is both diverse and dynamic. The second and third simulations contrast the performances of the three models in behavioral experiments, demonstrating advantages for the integrated model in accounting for behavioral data.

[1]  L M Reder,et al.  Awareness and working memory in strategy adaptivity , 2001, Memory & cognition.

[2]  Christopher W. Myers,et al.  Visual scan adaptation during repeated visual search. , 2010, Journal of vision.

[3]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[4]  John R. Anderson,et al.  History of Success and Current Context in Problem Solving Combined Influences on Operator Selection , 1996, Cognitive Psychology.

[5]  M. Chun,et al.  Contextual cueing of visual attention , 2022 .

[6]  John R. Anderson,et al.  Reflections of the Environment in Memory Form of the Memory Functions , 2022 .

[7]  John E. Laird,et al.  Soar-RL: integrating reinforcement learning with Soar , 2005, Cognitive Systems Research.

[8]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[9]  E. Tolman.  Cognitive maps in rats and men. , 1948, Psychological review.

[10]  B. Balleine,et al.  Motivational control of goal-directed action , 1994 .

[11]  John R. Anderson,et al.  From recurrent choice to skill learning: a reinforcement-learning model. , 2006, Journal of experimental psychology. General.

[12]  Christopher W. Myers,et al.  The insistence of vision: Why do people look at a salient stimulus when it signals target absence? , 2011 .

[13]  Frank J. Lee,et al.  Production Compilation: A Simple Mechanism to Model Complex Skill Acquisition , 2003, Hum. Factors.

[14]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[15]  Frank E. Ritter,et al.  The Rise of Cognitive Architectures , 2007, Integrated Models of Cognitive Systems.

[16]  D. Quartermain,et al.  Incidental learning in a simple task. , 1960 .

[17]  Wayne D. Gray.  Composition and Control of Integrated Cognitive Systems , 2007, Integrated Models of Cognitive Systems.

[18]  Wai-Tat Fu,et al.  SNIF-ACT: A Cognitive Model of User Navigation on the World Wide Web , 2007, Hum. Comput. Interact..

[19]  P. Dayan,et al.  Model-based influences on humans’ choices and striatal prediction errors , 2011, Neuron.

[20]  Marsha C. Lovett,et al.  Task representations, strategy variability, and base-rate neglect , 1999 .

[21]  M. Pomplun,et al.  Distractor Ratio Influences Patterns of Eye Movements during Visual Search , 2000, Perception.

[22]  R. Siegler,et al.  Conscious and unconscious strategy discoveries: a microgenetic analysis. , 1998, Journal of experimental psychology. General.

[23]  H. Blodgett,et al.  The effect of the introduction of reward upon the maze performance of rats , 1929 .

[24]  Kevin A. Gluck,et al.  Cognitive Architectures for Human Factors in Aviation , 2010 .

[25]  P. Dayan,et al.  States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning , 2010, Neuron.

[26]  Stefan Schaal,et al.  Reinforcement Learning for Humanoid Robotics , 2003 .

[27]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[28]  John R. Anderson.  How Can the Human Mind Occur in the Physical Universe , 2007 .

[29]  John P. Spencer,et al.  Autonomy in Action: Linking the Act of Looking to Memory Formation in Infancy via Dynamic Neural Fields , 2013, Cogn. Sci..

[30]  John R. Anderson,et al.  Dual learning processes in interactive skill acquisition. , 2008, Journal of experimental psychology. Applied.

[31]  Marsha C. Lovett,et al.  The importance of frameworks for directing empirical questions: reply to Goodie and Fantino (2000). , 2000, Journal of experimental psychology. General.

[32]  John R. Anderson,et al.  Rules of the Mind , 1993 .

[33]  H W STEVENSON.  Latent learning in children. , 1954, Journal of experimental psychology.

[34]  Vladislav Daniel Veksler,et al.  Goal-Proximity Decision-Making , 2013, Cogn. Sci..

[35]  A. Newell.  You can't play 20 questions with nature and win : projective comments on the papers of this symposium , 1973 .

[36]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[37]  Stellan Ohlsson,et al.  Effects of Multiple Learning Mechanisms in a Cognitive Architecture , 2011, CogSci.

[38]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[39]  N. Schmajuk,et al.  Latent learning, shortcuts and detours: a computational model , 2002, Behavioural Processes.

[40]  G. Stratton.  University of California publications in psychology , 1976 .

[41]  David E. Kieras,et al.  A computational theory of executive cognitive processes and multiple-task performance: Part 2. Accounts of psychological refractory-period phenomena. , 1997 .

[42]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.