An embodied account of serial order: How instabilities drive sequence generation

Learning and generating serially ordered sequences of actions is a core component of cognition both in organisms and in artificial cognitive systems. When these systems are embodied and situated in partially unknown environments, specific constraints arise for any neural mechanism of sequence generation. In particular, sequential action must resist fluctuating sensory information and be capable of generating sequences in which the individual actions may vary unpredictably in duration. We provide a solution to this problem within the framework of Dynamic Field Theory by proposing an architecture in which dynamic neural networks create stable states at each stage of a sequence. These neural attractors are destabilized in a cascade of bifurcations triggered by a neural representation of a condition of satisfaction for each action. We implement the architecture on a robotic vehicle in a color search task, demonstrating both sequence learning and sequence generation on the basis of low-level sensory information.

[1]  L. A. Jeffress,et al.  Cerebral Mechanisms in Behavior , 1953 .

[2]  T. Shallice,et al.  CONTENTION SCHEDULING AND THE CONTROL OF ROUTINE ACTIVITIES , 2000, Cognitive neuropsychology.

[3]  Lev S Tsimring,et al.  Dynamics-based sequential memory: winnerless competition of patterns. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[4]  Ramón Huerta,et al.  Dynamics of sequential decision making. , 2006, Physical review letters.

[5]  Edmund T. Rolls,et al.  Learning movement sequences with a delayed reward signal in a hierarchical model of motor function , 2007, Neural Networks.

[6]  J. Tanji.  Sequential organization of multiple movements: involvement of cortical motor areas. , 2001, Annual review of neuroscience.

[7]  D. Plaut,et al.  Doing without schema hierarchies: a recurrent connectionist approach to normal and impaired routine sequential action. , 2004, Psychological review.

[8]  T. Mexia,et al.  Author ' s personal copy , 2009 .

[9]  Chris Mellish,et al.  Current research in natural language generation , 1990 .

[10]  Daeyeol Lee,et al.  Prefrontal Neural Correlates of Memory for Sequences , 2007, The Journal of Neuroscience.

[11]  Toby Tyrell,et al.  The use of hierarchies for action selection , 1993 .

[12]  J. Yelnik.  Functional anatomy of the basal ganglia , 2002, Movement disorders : official journal of the Movement Disorder Society.

[13]  Stephen Grossberg,et al.  SOVEREIGN: An autonomous neural system for incrementally learning planned action sequences to navigate towards a rewarded goal , 2008, Neural Networks.

[14]  M. Merello,et al.  [Functional anatomy of the basal ganglia]. , 2000, Revista de neurologia.

[15]  O. Hikosaka,et al.  Role of the basal ganglia in the control of purposive saccadic eye movements. , 2000, Physiological reviews.

[16]  Thomas Wennekers,et al.  Dynamic Approximation of Spatiotemporal Receptive Fields in Nonlinear Neural Field Models , 2002, Neural Computation.

[17]  Alexander Riegler,et al.  When is a cognitive system embodied? , 2002, Cognitive Systems Research.

[18]  Stephen Grossberg,et al.  A Theory of Human Memory: Self-Organization and Performance of Sensory-Motor Codes, Maps, and Plans , 1982 .

[19]  B. Ermentrout.  Neural networks as spatio-temporal pattern-forming systems , 1998 .

[20]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[21]  George Houghton,et al.  Parallel Models of Serial Behaviour: Lashley Revisited , 1995 .

[22]  Günther Palm,et al.  Syntactic sequencing in Hebbian cell assemblies , 2009, Cognitive Neurodynamics.

[23]  J. Searle.  Intentionality: An Essay in the Philosophy of Mind , 1983 .

[24]  G. E. Alexander,et al.  Movement sequence-related activity reflecting numerical order of components in supplementary and presupplementary motor areas. , 1998, Journal of neurophysiology.

[25]  Thomas P. Trappenberg,et al.  Self-organizing continuous attractor networks and motor function , 2003, Neural Networks.

[26]  Peter Ford Dominey,et al.  A Model of Corticostriatal Plasticity for Learning Oculomotor Associations and Sequences , 1995, Journal of Cognitive Neuroscience.

[27]  Axel Steinhage,et al.  Dynamical systems for the behavioral organization of autonomous robot navigation , 1998, Other Conferences.

[28]  Gustavo Deco,et al.  Sequential Memory: A Putative Neural and Synaptic Dynamical Mechanism , 2005, Journal of Cognitive Neuroscience.

[29]  Thomas Wennekers,et al.  Operational Cell Assemblies as a Paradigm for Brain-Inspired Future Computing Architectures , 2006 .

[30]  R E Hampson,et al.  Ensemble Activity and Behavior—What's the Code? , 1995, Science.

[31]  K. Doya,et al.  Parallel neural networks for learning sequential procedures , 1999, Trends in Neurosciences.

[32]  A. Graybiel,et al.  Representation of Action Sequence Boundaries by Macaque Prefrontal Cortical Neurons , 2003, Science.

[33]  David W. Payton,et al.  Plan guided reaction , 1990, IEEE Trans. Syst. Man Cybern..

[34]  K. Lashley.  The problem of serial order in behavior , 1951 .

[35]  S. Lewandowsky,et al.  An endogenous distributed model of ordering in serial recall , 2002, Psychonomic bulletin & review.

[36]  Robert Rosen,et al.  Progress in Theoretical Biology , 2012 .

[37]  Gregor Schöner,et al.  Dynamics of behavior: Theory and applications for autonomous robot architectures , 1995, Robotics Auton. Syst..

[38]  Gregor Schöner,et al.  Dynamic Field Theory and Embodied Communication , 2006, ZiF Workshop.

[39]  A. Georgopoulos,et al.  Motor cortical encoding of serial order in a context-recall task. , 1999, Science.

[40]  Tim Shallice,et al.  Neuropsychologically plausible sequence generation in a multi-layer network model of spelling , 1999 .

[41]  Ronald C. Arkin,et al.  Temporal coordination of perceptual algorithms for mobile robot navigation , 1994, IEEE Trans. Robotics Autom..

[42]  P. Maes.  How to Do the Right Thing , 1989 .

[43]  A. Georgopoulos.  Higher order motor control. , 1991, Annual review of neuroscience.

[44]  H. Markram,et al.  Coding and learning of behavioral sequences , 2004, Trends in Neurosciences.

[45]  R. Henson.  Short-Term Memory for Serial Order: The Start-End Model , 1998, Cognitive Psychology.

[46]  E. Procyk,et al.  Anterior cingulate activity during routine and non-routine sequential behaviors in macaques , 2000, Nature Neuroscience.

[47]  S. Amari.  Dynamics of pattern formation in lateral-inhibition type neural fields , 1977, Biological Cybernetics.

[48]  J. Adams.  Learning of movement sequences , 1984 .

[49]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[50]  Thomas P. Trappenberg,et al.  Self-organising continuous attractor networks with multiple activity packets, and the representation of space , 2004, Neural Networks.

[51]  N. Burgess.  Memory for Serial Order : A Network Model of the Phonological Loop and its Timing , 1999 .

[52]  Howard Eichenbaum,et al.  Comparative cognition, hippocampal function, and recollection , 2006 .

[53]  L. Hazrati,et al.  Functional anatomy of the basal ganglia , 1995 .

[54]  Aude Billard,et al.  From Animals to Animats , 2004 .

[55]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[56]  George Houghton,et al.  The problem of serial order: a neural network model of sequence learning and recall , 1990 .

[57]  Karl J. Friston,et al.  The Dynamic Brain: From Spiking Neurons to Neural Masses and Cortical Fields , 2008, PLoS Comput. Biol..

[58]  Caroline Palmer,et al.  Speed, Accuracy, and Serial Order in Sequence Production , 2007, Cogn. Sci..

[59]  Estela Bicho,et al.  The dynamic approach to autonomous robotics demonstrated on a low-level vehicle platform , 1997, Robotics Auton. Syst..

[60]  George Houghton,et al.  Serial order and consonant–vowel structure in a graphemic output buffer model , 2005, Brain and Language.

[61]  G D Brown,et al.  Oscillator-based memory for serial order. , 2000, Psychological review.

[62]  G. Schöner.  The Cambridge Handbook of Computational Psychology: Dynamical Systems Approaches to Cognition , 2008 .

[63]  D. Norris,et al.  The primacy model: a new model of immediate serial recall. , 1998, Psychological review.

[64]  Jeffrey S. Johnson,et al.  Moving to higher ground: The dynamic field theory and the dynamics of visual cognition. , 2008, New ideas in psychology.

[65]  J. W. Aldridge,et al.  Coding of Serial Order by Neostriatal Neurons: A “Natural Action” Approach to Movement Sequence , 1998, The Journal of Neuroscience.

[66]  Gary S. Dell,et al.  Connectionist models of language production: lexical access and grammatical encoding , 1999, Cogn. Sci..

[67]  Christian Faubel,et al.  Learning to recognize objects on the fly: A neurally based dynamic field approach , 2008, Neural Networks.

[68]  Max Coltheart,et al.  Cognitive Neuropsychology , 2014, Scholarpedia.

[69]  T. Hartley,et al.  A Linguistically Constrained Model of Short-Term Memory for Nonwords ☆ , 1996 .

[70]  J. Houk,et al.  Model of cortical-basal ganglionic processing: encoding the serial order of sensory events. , 1998, Journal of neurophysiology.

[71]  Roland Potthast,et al.  Dimensional Reduction for the Inverse Problem of Neural Field Theory , 2009, Front. Comput. Neurosci..

[72]  Ruzena Bajcsy,et al.  Discrete Event Systems for autonomous mobile agents , 1994, Robotics Auton. Syst..

[73]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[74]  J. Cowan,et al.  A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue , 1973, Kybernetik.

[75]  J. Tanji,et al.  Role for cingulate motor area cells in voluntary movement selection based on reward. , 1998, Science.

[76]  S. Grossberg,et al.  Laminar cortical dynamics of cognitive and motor working memory, sequence learning and performance: toward a unified theory of how the cerebral cortex works. , 2008, Psychological review.

[77]  Ipke Wachsmuth,et al.  Modelling Communication with Robots and Virtual Humans , 2008 .

[78]  Matthew M Botvinick,et al.  Short-term memory for serial order: a recurrent neural network model. , 2006, Psychological review.