Such stuff as habits are made on: A reply to Cooper and Shallice (2006).

The representations and mechanisms guiding everyday routine sequential action remain incompletely understood. In recent work, the authors proposed a computational model of routine sequential behavior that took the form of a recurrent neural network (M. Botvinick & D. C. Plaut, 2004). Subsequently, R. P. Cooper and T. Shallice (2006) put forth a detailed critique of that work, contrasting it with their own account, which assumes a strict hierarchical processing system (R. P. Cooper & T. Shallice, 2000). The authors respond here to the main points of R. P. Cooper and T. Shallice’s (2006) critique. Although careful and constructive, the arguments offered by R. P. Cooper and T. Shallice (2006) mistook several superficial implementational issues for fundamental theoretical ones, underestimated the computational power of recurrent networks as a class, and in some ways mischaracterized the relationship between the accounts they compare. In responding to these points, the authors articulate several key theoretical choices facing models of routine sequential behavior.

[1]  D. Spalding.  The Principles of Psychology , 1873, Nature.

[2]  HighWire Press.  Philosophical Transactions of the Royal Society of London , 1781, The London Medical Journal.

[3]  G. Miller,et al.  Plans and the structure of behavior , 1960 .

[4]  Eugene Galanter,et al.  Handbook of mathematical psychology: I. , 1963 .

[5]  A. Battersby.  Plans and the Structure of Behavior , 1968 .

[6]  J. Shaoul.  Human Error , 1973, Nature.

[7]  G. Schwartz,et al.  Consciousness and Self-Regulation , 1976 .

[8]  Richard C. Anderson,et al.  Schooling and the Acquisition of Knowledge , 1978 .

[9]  D. Norman,et al.  Attention to action: Willed and automatic control , 1980 .

[10]  D. Norman,et al.  Attention to Action: Willed and Automatic Control of Behavior Technical Report No. 8006. , 1980 .

[11]  O. Andy.  The prefrontal cortex: Anatomy, physiology and neuropsychology of the frontal lobe , 1981 .

[12]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[13]  D. Norman.  Categorization of action slips. , 1981 .

[14]  A. Dickinson.  Actions and habits: the development of behavioural autonomy , 1985 .

[15]  Geoffrey E. Hinton,et al.  Schemata and Sequential Thought Processes in PDP Models , 1986 .

[16]  Donald A. Norman,et al.  Attention to Action , 1986 .

[17]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[18]  Philip E. Agre,et al.  The dynamic structure of everyday life , 1988 .

[19]  Roger C. Schank,et al.  SCRIPTS, PLANS, GOALS, AND UNDERSTANDING , 1988 .

[20]  M. W. Montgomery,et al.  The Quantitative Description of Action Disorganisation after Brain Damage: A Case Study , 1991 .

[21]  Axel Cleeremans,et al.  Mechanisms of Implicit Learning: Connectionist Models of Sequence Processing , 1993 .

[22]  Y. Kareev,et al.  The emergence of events , 1994, Cognition.

[23]  Michael C. Mozer,et al.  Neural Network Music Composition by Prediction: Exploring the Benefits of Psychoacoustic Constraints and Multi-scale Processing , 1994, Connect. Sci..

[24]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[25]  JORDAN GRAFMAN,et al.  Similarities and Distinctions among Current Models of Prefrontal Cortical Functions , 1995, Annals of the New York Academy of Sciences.

[26]  Michael W. Montgomery,et al.  The quantitative description of action disorganisation after brain damage: a case study , 1991 .

[27]  James L. McClelland,et al.  Understanding normal and impaired word reading: computational principles in quasi-regular domains. , 1996, Psychological review.

[28]  Keith J. Holyoak,et al.  Structure and Functions of the Human Prefrontal Cortex , 1996 .

[29]  Jonathan D. Cohen,et al.  A preliminary theory of the interactions between prefrontal cortex and hippocampus that contribute to planning and prospective memory. , 1996 .

[30]  J. Fuster.  The Prefrontal Cortex , 1997 .

[31]  Michael W. Montgomery,et al.  Naturalistic action impairment in closed head injury. , 1998, Neuropsychology.

[32]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[33]  B. Balleine,et al.  Goal-directed instrumental action: contingency and incentive learning and their cortical substrates , 1998, Neuropharmacology.

[34]  J. W. Aldridge,et al.  Coding of Serial Order by Neostriatal Neurons: A “Natural Action” Approach to Movement Sequence , 1998, The Journal of Neuroscience.

[35]  L. Buxbaum,et al.  Ideational apraxia and naturalistic action. , 1998, Cognitive neuropsychology.

[36]  Doina Precup,et al.  Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning , 1999, Artif. Intell..

[37]  R. French.  Catastrophic forgetting in connectionist networks , 1999, Trends in Cognitive Sciences.

[38]  Paul Rodríguez,et al.  A Recurrent Neural Network that Learns to Count , 1999, Connect. Sci..

[39]  Carol A. Seger,et al.  Striatal activation during acquisition of a cognitive skill. , 1999, Neuropsychology.

[40]  T. Shallice,et al.  CONTENTION SCHEDULING AND THE CONTROL OF ROUTINE ACTIVITIES , 2000, Cognitive neuropsychology.

[41]  G. Dell,et al.  Speech errors, phonotactic constraints, and implicit learning: a study of the role of experience in language production. , 2000, Journal of experimental psychology. Learning, memory, and cognition.

[42]  Jeffrey M. Zacks,et al.  Event structure in perception and conception. , 2001, Psychological bulletin.

[43]  J. Saffran.  The Use of Predictive Dependencies in Language Learning , 2001 .

[44]  J. Gregory Trafton,et al.  Memory for goals: an activation-based model , 2002, Cogn. Sci..

[45]  René Boel,et al.  Discrete event dynamic systems: Theory and applications. , 2002 .

[46]  Jürgen Schmidhuber,et al.  Finding temporal structure in music: blues improvisation with LSTM recurrent networks , 2002, Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing.

[47]  The Cognitive Neuropsychology of Everyday Actions , 2002, Neurocase.

[48]  James L. McClelland,et al.  Rules or connections in past-tense inflections: what does the evidence rule out? , 2002, Trends in Cognitive Sciences.

[49]  D. Plaut,et al.  Representing task context: proposals based on a connectionist model of action , 2002, Psychological research.

[50]  Sridhar Mahadevan,et al.  Recent Advances in Hierarchical Reinforcement Learning , 2003, Discret. Event Dyn. Syst..

[51]  John Millar Carroll.  HCI Models, Theories, and Frameworks: Toward a Multidisciplinary Science , 2003 .

[52]  B. John.  Information Processing and Skilled Behavior , 2003 .

[53]  Bonnie E. John,et al.  CHAPTER 4 – Information Processing and Skilled Behavior , 2003 .

[54]  R. Cooper.  Mechanisms for the generation and regulation of sequential behaviour , 2003 .

[55]  Sridhar Mahadevan,et al.  Recent Advances in Hierarchical Reinforcement Learning , 2003, Discret. Event Dyn. Syst..

[56]  E. Maguire,et al.  The Well-Worn Route and the Path Less Traveled Distinct Neural Bases of Route Following and Wayfinding in Humans , 2003, Neuron.

[57]  Robert M. French,et al.  Self-refreshing memory in artificial neural networks: learning temporal sequences without catastrophic forgetting , 2004, Connect. Sci..

[58]  Nuttapong Chentanez,et al.  Intrinsically Motivated Learning of Hierarchical Collections of Skills , 2004 .

[59]  B. Balleine,et al.  Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning , 2004, The European journal of neuroscience.

[60]  Emilio Salinas,et al.  Fast Remapping of Sensory Stimuli onto Motor Actions on the Basis of Contextual Modulation , 2004, The Journal of Neuroscience.

[61]  John R. Anderson,et al.  Skill Acquisition : Compilation of Weak-Method Problem Solutions , 2004 .

[62]  D. Plaut,et al.  Doing without schema hierarchies: a recurrent connectionist approach to normal and impaired routine sequential action. , 2004, Psychological review.

[63]  Jonathan D. Cohen,et al.  Prefrontal cortex and flexible cognitive control: rules without symbols. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[64]  Yael Niv,et al.  Uncertainty-based competition between prefrontal and striatal systems for behavioural control , 2005 .

[65]  Lauren M. Bylsma,et al.  Distraction and action slips in an everyday task: Evidence for a dynamic representation of task context , 2005, Psychonomic bulletin & review.

[66]  T. Shallice,et al.  The simulation of action disorganisation in complex activities of daily living , 2005, Cognitive neuropsychology.

[67]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[68]  Wei Zhang,et al.  A Computational Model of Eye Movements during Object Class Detection , 2005, NIPS.

[69]  T. Shallice,et al.  Structured representations in the control of behavior cannot be so easily dismissed: a reply to Botvinick and Plaut (2006) , 2006 .

[70]  D. Luce,et al.  Detection and Recognition " ' , 2006 .

[71]  T. Shallice,et al.  Hierarchical schemas and goals in the control of sequential behavior. , 2006, Psychological review.

[72]  R. Cooper.  Tool Use And Related Errors in Ideational Apraxia: the Quantitative Simulation of Patient Error Profiles , 2007, Cortex.