Doing without schema hierarchies: a recurrent connectionist approach to normal and impaired routine sequential action.

In everyday tasks, selecting actions in the proper sequence requires a continuously updated representation of temporal context. Previous models have addressed this problem by positing a hierarchy of processing units, mirroring the roughly hierarchical structure of naturalistic tasks themselves. The present study considers an alternative framework, in which the representation of context depends on recurrent connections within a network mapping from environmental inputs to actions. The ability of this approach to account for human performance was evaluated by applying it, through simulation, to a specific everyday task. The resulting model learned to deal flexibly with a complex set of sequencing constraints, encoding contextual information at multiple time scales within a single, distributed internal representation. Degrading this representation led to errors resembling those observed both in everyday behavior and in apraxia. Analysis of the model's function yielded numerous predictions relevant to both normal and apraxic performance.

[1]  D. Spalding The Principles of Psychology , 1873, Nature.

[2]  HighWire Press Philosophical Transactions of the Royal Society of London , 1781, The London Medical Journal.

[3]  Joseph Jastrow,et al.  The lapses of consciousness. , 1906 .

[4]  L. A. Jeffress Cerebral mechanisms in behavior : the Hixon symposium , 1951 .

[5]  K. Lashley The problem of serial order in behavior , 1951 .

[6]  G. Miller,et al.  Plans and the structure of behavior , 1960 .

[7]  J. O. Urmson,et al.  The William James Lectures , 1963 .

[8]  A. Battersby Plans and the Structure of Behavior , 1968 .

[9]  G. Miller,et al.  Plans and the structure of behavior , 1960 .

[10]  A. W. Melton,et al.  Coding Processes in Human Memory. , 1973 .

[11]  J. Shaoul Human Error , 1973, Nature.

[12]  C. Eriksen,et al.  Effects of noise letters upon the identification of a target letter in a nonsearch task , 1974 .

[13]  R. Davidson,et al.  Consciousness and Self-Regulation: Advances in Research and Theory IV , 1976 .

[14]  Wayne D. Gray,et al.  Basic objects in natural categories , 1976, Cognitive Psychology.

[15]  Daniel G. Bobrow,et al.  Descriptions: An intermediate stage in memory retrieval , 1979, Cognitive Psychology.

[16]  J. Reason Actions not as planned : The price of automatisation , 1979 .

[17]  Joseph S. Silverman,et al.  Consciousness and Self-Regulation: Advances in Research and Theory, Vol. 2 , 1979 .

[18]  D. Norman,et al.  Attention to action: Willed and automatic control , 1980 .

[19]  D. Norman,et al.  Attention to Action: Willed and Automatic Control of Behavior Technical Report No. 8006. , 1980 .

[20]  K. Poeck,et al.  A Disturbance in the Conceptual Organization of Actions in Patients with Ideational Apraxia , 1981, Cortex.

[21]  D. Norman Categorization of action slips. , 1981 .

[22]  A. W. Ellis Normality and pathology in cognitive functions , 1982 .

[23]  Donald A. Norman,et al.  Simulating a Skilled Typist: A Study of Skilled Cognitive-Motor Performance , 1982, Cogn. Sci..

[24]  Roger C. Schank,et al.  Dynamic memory - a theory of reminding and learning in computers and people , 1983 .

[25]  C SchankRoger,et al.  Dynamic Memory: A Theory of Reminding and Learning in Computers and People , 1983 .

[26]  Raja Parasuraman,et al.  Varieties of attention , 1984 .

[27]  M. Albert,et al.  Varieties of perseveration , 1984, Neuropsychologia.

[28]  P. Greenfield A theory of the teacher in the learning activities of everyday life. , 1984 .

[29]  S. Ullman Visual routines , 1984, Cognition.

[30]  D. G. MacKay A Theory of The Representation, Organization And Timing of Action With Implications For Sequencing Disorders , 1985 .

[31]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[32]  Michael Georgeff,et al.  Reasoning about actions and plans , 1986 .

[33]  Eileen C. Schwab,et al.  Pattern recognition by humans and machines , 1986 .

[34]  James L. McClelland,et al.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition : Psychological and Biological Models , 1986 .

[35]  J. Duncan Disorganisation of behaviour after frontal lobe damage , 1986 .

[36]  Geoffrey E. Hinton,et al.  Schemata and Sequential Thought Processes in PDP Models , 1986 .

[37]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[38]  L. Henry,et al.  Everyday Memory, Actions, and Absent-Mindedness , 1986 .

[39]  John R. Anderson,et al.  Skill Acquisition: Compilation of Weak-Method Problem Solutions. , 1987 .

[40]  A. Allport,et al.  Selection for action: Some behavioral and neurophysiological considerations of attention and action , 1987 .

[41]  S. Grossberg The Adaptive Self-Organization of Serial Order in Behavior: Speech, Language, And Motor Control , 1987 .

[42]  Allen Newell,et al.  SOAR: An Architecture for General Intelligence , 1987, Artif. Intell..

[43]  Philip E. Agre,et al.  ABSTRACT REASONING AS EMERGENT FROM CONCRETE ACTIVITY , 1987 .

[44]  Mark C. Detweiler,et al.  The Role of Practice in Dual-Task Performance: Toward Workload Modeling a Connectionist/Control Architecture , 1988, Human factors.

[45]  Philip E. Agre,et al.  The dynamic structure of everyday life , 1988 .

[46]  Roger C. Schank,et al.  SCRIPTS, PLANS, GOALS, AND UNDERSTANDING , 1988 .

[47]  James L. McClelland,et al.  Learning Subsequential Structure in Simple Recurrent Networks , 1988, NIPS.

[48]  Richard A. Andersen,et al.  A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons , 1988, Nature.

[49]  G. F. Tremblay,et al.  The Prefrontal Cortex , 1989, Neurology.

[50]  Z. Pylyshyn The role of location indexes in spatial perception: A sketch of the FINST spatial-index model , 1989, Cognition.

[51]  H. Heuer,et al.  Perspectives on Perception and Action , 1989 .

[52]  Barak A. Pearlmutter Learning State Space Trajectories in Recurrent Neural Networks , 1989, Neural Computation.

[53]  Richard J. Brown Neuropsychology Mental Structure , 1989 .

[54]  Geoffrey E. Hinton Connectionist Learning Procedures , 1989, Artif. Intell..

[55]  James L. McClelland,et al.  Sentence comprehension: A parallel distributed processing approach , 1989, Language and Cognitive Processes.

[56]  Michael McCloskey,et al.  Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem , 1989 .

[57]  Chris Mellish,et al.  Current research in natural language generation , 1990 .

[58]  Geoffrey E. Hinton,et al.  Distributed Representations , 1986, The Philosophy of Artificial Intelligence.

[59]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[60]  Joanne L. Miller,et al.  Speech Perception , 1990, Springer Handbook of Auditory Research.

[61]  Ann L. Brown,et al.  Domain-Specific Principles Affect Learning and Transfer in Children , 1990, Cogn. Sci..

[62]  L. Squire,et al.  The Neuropsychology of Memory , 1990 .

[63]  George Houghton,et al.  The problem of serial order: a neural network model of sequence learning and recall , 1990 .

[64]  Dana H. Ballard,et al.  Active Perception and Reinforcement Learning , 1990, Neural Computation.

[65]  Michael I. Jordan Attractor dynamics and parallelism in a connectionist sequential machine , 1990 .

[66]  James L. McClelland,et al.  On the control of automatic processes: a parallel distributed processing account of the Stroop effect. , 1990, Psychological review.

[67]  M. W. Montgomery,et al.  The Quantitative Description of Action Disorganisation after Brain Damage: A Case Study , 1991 .

[68]  Colin M. Macleod Half a century of research on the Stroop effect: an integrative review. , 1991, Psychological bulletin.

[69]  Richard Reviewer-Granger Unified Theories of Cognition , 1991, Journal of Cognitive Neuroscience.

[70]  James L. McClelland,et al.  Learning the structure of event sequences. , 1991, Journal of experimental psychology. General.

[71]  Larry Wall,et al.  Programming Perl , 1991 .

[72]  E. Friedman,et al.  Temporal processing. , 1991, Journal of learning disabilities.

[73]  B. Baars Experimental slips and human error : exploring the architecture of volition , 1992 .

[74]  J. Cohen,et al.  Context, cortex, and dopamine: a connectionist approach to behavior and biology in schizophrenia. , 1992, Psychological review.

[75]  J. Reason Cognitive Underspecification Its Variety and Consequences , 1992 .

[76]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[77]  D H Ballard,et al.  Hand-eye coordination during sequential tasks. , 1992, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[78]  Susan J. Lederman,et al.  Cognitive representations of functional interactions with objects , 1993, Memory & cognition.

[79]  Gary S. Dell,et al.  Structure and Content in Language Production: A Theory of Frame Constraints in Phonological Speech Errors , 1993, Cogn. Sci..

[80]  Michael W. Montgomery,et al.  Cognitive theory and the study of everyday action disorders after brain damage , 1993 .

[81]  Alan D. Baddeley,et al.  Attention: Selection, Awareness, and Control , 1993 .

[82]  J. Elman Learning and development in neural networks: the importance of starting small , 1993, Cognition.

[83]  Y. Kareev,et al.  The emergence of events , 1994, Cognition.

[84]  G. Dell,et al.  Inhibition in interactive activation models of linguistic selection and sequencing. , 1994 .

[85]  Michael I. Jordan,et al.  A Model of the Learning of Arm Trajectories from Spatial Deviations , 1994, Journal of Cognitive Neuroscience.

[86]  T. Carr,et al.  Inhibitory Processes in Attention, Memory and Language , 1994 .

[87]  Lorraine K. Tyler,et al.  A Distributed Memory Model of the Associative Boost in Semantic Priming , 1994, Connect. Sci..

[88]  S. Kosslyn Image and Brain , 1994 .

[89]  G. Kane Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[90]  Garrison W. Cottrell,et al.  Acquiring the Mapping from Meaning to Sounds , 1994, Connect. Sci..

[91]  Yves Chauvin,et al.  Backpropagation: the basic theory , 1995 .

[92]  Ronald J. Williams,et al.  Gradient-based learning algorithms for recurrent networks and their computational complexity , 1995 .

[93]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[94]  George Houghton,et al.  Parallel Models of Serial Behaviour: Lashley Revisited , 1995 .

[95]  Michael W. Montgomery,et al.  The quantitative description of action disorganisation after brain damage: a case study , 1991 .

[96]  MYRNA F. SCHWARTZ,et al.  Re‐examining the Role of Executive Functions in Routine Action Production a , 1995, Annals of the New York Academy of Sciences.

[97]  K. A. Ericsson,et al.  Long-term working memory. , 1995, Psychological review.

[98]  Michael W. Montgomery,et al.  Analysis of a disorder of everyday action , 1995 .

[99]  Long Ji Lin,et al.  Reinforcement Learning of Non-Markov Decision Processes , 1995, Artif. Intell..

[100]  William D. Marslen-Wilson,et al.  A Connectionist Model of Phonological Representation in Speech Perception , 1995, Cogn. Sci..

[101]  Andrew McCallum,et al.  Reinforcement learning with selective perception and hidden state , 1996 .

[102]  J. Duncan Cooperating brain systems in selective perception and action. , 1996 .

[103]  Catherine Hanson,et al.  Development of Schemata during Event Parsing: Neisser's Perceptual Cycle as a Recurrent Connectionist Network , 1996, Journal of Cognitive Neuroscience.

[104]  James M. Bower,et al.  Computational Neuroscience: Trends in Research , 1996 .

[105]  Keith J. Holyoak,et al.  Structure and Functions of the Human Prefrontal Cortex , 1996 .

[106]  James L. McClelland,et al.  Information integration in perception and communication , 1996 .

[107]  Jonathan D. Cohen,et al.  A preliminary theory of the interactions between prefrontal cortex and hippocampus that contribute to planning and prospective memory. , 1996 .

[108]  Michael C. Mozer,et al.  Mathematical Perspectives on Neural Networks , 1996 .

[109]  Randall C. O'Reilly,et al.  Biologically Plausible Error-Driven Learning Using Local Activation Differences: The Generalized Recirculation Algorithm , 1996, Neural Computation.

[110]  D. Norris,et al.  THE QUARTERLY JOURNAL OF EXPERIMENTAL PSYCHOLOGY, 1996, 49A (1), 80 ± 115 Unchained Memory: Error Patterns Rule out Chaining Models of Immediate Serial Recall , 2022 .

[111]  G. Dell,et al.  Language production and serial order: a functional analysis and a model. , 1997, Psychological review.

[112]  D. Plaut,et al.  Learning in Dynamic Decision Tasks: Computational Model and Empirical Evidence , 1997 .

[113]  James L. McClelland,et al.  Rethinking infant knowledge: toward an adaptive process account of successes and failures in object permanence tasks. , 1997, Psychological review.

[114]  Rajesh P. N. Rao,et al.  Embodiment is the foundation, not a level , 1996, Behavioral and Brain Sciences.

[115]  Gordon D. A. Brown,et al.  To Repeat or Not to Repeat: The Time Course of Response Suppression in Sequential Behaviour , 1997, NCPW.

[116]  William B. Levy,et al.  Predicting novel paths to goals by a simple, biologically inspired neural network , 1997 .

[117]  Michael I. Jordan Serial Order: A Parallel Distributed Processing Approach , 1997 .

[118]  Michael W. Montgomery,et al.  Naturalistic action impairment in closed head injury. , 1998, Neuropsychology.

[119]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[120]  Morten H. Christiansen,et al.  Learning to Segment Speech Using Multiple Cues: A Connectionist Model , 1998 .

[121]  B. Habibi,et al.  Pengi : An Implementation of A Theory of Activity , 1998 .

[122]  T. Sejnowski,et al.  A Computational Model of How the Basal Ganglia Produce Sequences , 1998, Journal of Cognitive Neuroscience.

[123]  L. Buxbaum,et al.  Ideational apraxia and naturalistic action. , 1998, Cognitive neuropsychology.

[124]  P. Ashworth,et al.  Handbook of Implicit Learning , 1998 .

[125]  G. Humphreys,et al.  Visual affordances direct action: neuropsychological evidence from manual interference. , 1998, Cognitive neuropsychology.

[126]  G. Humphreys,et al.  Disordered action schema and action disorganisation syndrome , 1998 .

[127]  R. Ellis,et al.  On the relations between seen objects and components of potential actions. , 1998, Journal of experimental psychology. Human perception and performance.

[128]  Peter Ford Dominey,et al.  A shared system for learning serial and temporal structure of sensori-motor sequences? Evidence from simulation and human experiments. , 1998, Brain research. Cognitive brain research.

[129]  David C. Plaut,et al.  The emergence of phonology from the interplay of speech comprehension and production ; A distributed connectionist approach , 1998 .

[130]  Gary S. Dell,et al.  Connectionist models of language production: lexical access and grammatical encoding , 1999, Cogn. Sci..

[131]  T. Rohde LENS : The light , efficient network simulator , 1999 .

[132]  Brian MacWhinney,et al.  The emergence of language. , 1999 .

[133]  B. Rogoff,et al.  Everyday Cognition: Development in Social Context , 1999 .

[134]  Nick Chater,et al.  Toward a connectionist model of recursion in human linguistic performance , 1999, Cogn. Sci..

[135]  Douglas L. T. Rohde,et al.  Language acquisition in the absence of explicit negative evidence: how important is starting small? , 1999, Cognition.

[136]  Thesis Proposal,et al.  A Connectionist Model of Sentence Comprehension and Production , 1999 .

[137]  Mark S. Seidenberg,et al.  The emergence of grammaticality in connectionist networks. , 1999 .

[138]  Gary S. Dell,et al.  The emergence of language from serial order and procedural memory. , 1999 .

[139]  David C. Plaut,et al.  A connectionist approach to word reading and acquired dyslexia: extension to sequential processing , 1999, Cogn. Sci..

[140]  K. Doya,et al.  Parallel neural networks for learning sequential procedures , 1999, Trends in Neurosciences.

[141]  M. Hayhoe Vision Using Routines: A Functional Account of Vision , 2000 .

[142]  T. Shallice,et al.  CONTENTION SCHEDULING AND THE CONTROL OF ROUTINE ACTIVITIES , 2000, Cognitive neuropsychology.

[143]  M. Page,et al.  Connectionist modelling in psychology: A localist manifesto , 2000, Behavioral and Brain Sciences.

[144]  Gordon D. A. Brown,et al.  Serial Control of Phonology in Speech Production: A Hierarchical Model , 2000, Cognitive Psychology.

[145]  Paul Barry,et al.  Programming Perl 3rd Edition , 2000 .

[146]  G D Brown,et al.  Oscillator-based memory for serial order. , 2000, Psychological review.

[147]  R. Siegler,et al.  ACROSS THE GREAT DIVIDE : BRIDGING THE GAP BETWEEN UNDERSTANDING OF TODDLERS ' AND OLDER CHILDREN ' S THINKING Zhe , 2002 .

[148]  James L. McClelland,et al.  Stipulating versus discovering representations , 2000, Behavioral and Brain Sciences.

[149]  Glyn W. Humphreys,et al.  The organization of sequential actions , 2000 .

[150]  J. Driver,et al.  Control of Cognitive Processes: Attention and Performance XVIII , 2000 .

[151]  Peter Ford Dominey,et al.  Neural network processing of natural language: I. Sensitivity to serial, temporal and abstract structure of language in the infant , 2000 .

[152]  Michael J. Frank,et al.  Interactions between frontal cortex and basal ganglia in working memory: A computational model , 2001, Cognitive, affective & behavioral neuroscience.

[153]  Axel Cleeremans,et al.  Implicit learning out of the lab: the case of orthographic regularities. , 2001, Journal of experimental psychology. General.

[154]  Jeffrey M. Zacks,et al.  Event structure in perception and conception. , 2001, Psychological bulletin.

[155]  Sarah H. Creem,et al.  Grasping objects by their handles: a necessary interaction between cognition and action. , 2001, Journal of experimental psychology. Human perception and performance.

[156]  Jenny R Saffran,et al.  Words in a sea of sounds: the output of infant statistical learning , 2001, Cognition.

[157]  B. Hommel,et al.  Effect anticipation and action control. , 2001, Journal of experimental psychology. Human perception and performance.

[158]  J. Riddoch,et al.  Visual affordances and object selection , 2001 .

[159]  David C. Plaut,et al.  A connectionist model of sentence comprehension and production , 2002 .

[160]  Albert K. Lee,et al.  Memory of Sequential Experience in the Hippocampus during Slow Wave Sleep , 2002, Neuron.

[161]  H. Eichenbaum,et al.  Critical role of the hippocampus in memory for sequences of events , 2002, Nature Neuroscience.

[162]  Raymond P. Kesner,et al.  The role of the hippocampus in memory for the temporal order of a sequence of odors. , 2002 .

[163]  Paul E. Gilbert,et al.  The role of the hippocampus in memory for the temporal order of a sequence of odors. , 2002, Behavioral neuroscience.

[164]  D. Plaut,et al.  Representing task context: proposals based on a connectionist model of action , 2002, Psychological research.

[165]  John R. Anderson,et al.  Skill Acquisition : Compilation of Weak-Method Problem Solutions , 2004 .

[166]  K. Poeck Ideational apraxia , 2004, Journal of Neurology.

[167]  J. Elman Distributed representations, simple recurrent networks, and grammatical structure , 1991, Machine Learning.

[168]  Lauren M. Bylsma,et al.  Distraction and action slips in an everyday task: Evidence for a dynamic representation of task context , 2005, Psychonomic bulletin & review.

[169]  Philipp Slusallek,et al.  Introduction to real-time ray tracing , 2005, SIGGRAPH Courses.

[170]  James L. McClelland,et al.  Graded State Machines: The Representation of Temporal Contingencies in Simple Recurrent Networks , 2005, Machine Learning.

[171]  Donald G. MacKay,et al.  The organization of perception and action. A theory for language and other cognitive skills , 1987, The Italian Journal of Neurological Sciences.