Time Scale Hierarchies in the Functional Organization of Complex Behaviors

Traditional approaches to cognitive modelling generally portray cognitive events in terms of ‘discrete’ states (point attractor dynamics) rather than in terms of processes, thereby neglecting the time structure of cognition. In contrast, more recent approaches explicitly address this temporal dimension, but typically provide no entry points into cognitive categorization of events and experiences. With the aim to incorporate both these aspects, we propose a framework for functional architectures. Our approach is grounded in the notion that arbitrary complex (human) behaviour is decomposable into functional modes (elementary units), which we conceptualize as low-dimensional dynamical objects (structured flows on manifolds). The ensemble of modes at an agent’s disposal constitutes his/her functional repertoire. The modes may be subjected to additional dynamics (termed operational signals), in particular, instantaneous inputs, and a mechanism that sequentially selects a mode so that it temporarily dominates the functional dynamics. The inputs and selection mechanisms act on faster and slower time scales then that inherent to the modes, respectively. The dynamics across the three time scales are coupled via feedback, rendering the entire architecture autonomous. We illustrate the functional architecture in the context of serial behaviour, namely cursive handwriting. Subsequently, we investigate the possibility of recovering the contributions of functional modes and operational signals from the output, which appears to be possible only when examining the output phase flow (i.e., not from trajectories in phase space or time).

[1]  Andreas Daffertshofer,et al.  Benefits and Pitfalls in Analyzing Noise in Dynamical Systems - On Stochastic Differential Equations and System Identification , 2011, Nonlinear Dynamics in Human Behavior.

[2]  G. V. Galen,et al.  On the Simultaneous Processing of Words, Letters and Strokes in Handwriting: Evidence For a Mixed Linear and Parallel Model , 1986 .

[3]  M. Fayol,et al.  The effect of the number of syllables on handwriting production , 2008 .

[4]  Stephen Grossberg,et al.  Adaptive Resonance Theory , 2010, Encyclopedia of Machine Learning.

[5]  René Thom,et al.  Structural stability and morphogenesis , 1977, Pattern Recognit..

[6]  Viktor Jirsa,et al.  Complex Processes from Dynamical Architectures with Time-Scale Hierarchy , 2011, PloS one.

[7]  Stephen Monsell,et al.  The Latency and Duration of Rapid Movement Sequences: Comparisons of Speech and Typewriting , 1978 .

[8]  R. Jackendoff Foundations of Language: Brain, Meaning, Grammar, Evolution , 2002 .

[9]  J. Kelso,et al.  Cortical coordination dynamics and cognition , 2001, Trends in Cognitive Sciences.

[10]  K. Doya,et al.  A unifying computational framework for motor control and social interaction. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[11]  R. Ivry,et al.  The neural representation of time , 2004, Current Opinion in Neurobiology.

[12]  Viktor K. Jirsa,et al.  Connectivity and dynamics of neural information processing , 2007, Neuroinformatics.

[13]  H. Maturana,et al.  The Tree of Knowledge: The Biological Roots of Human Understanding , 2007 .

[14]  J. Kelso,et al.  Phase transitions in spatiotemporal patterns of brain activity and behavior , 1995 .

[15]  H. Haken,et al.  The impact of fluctuations on the recognition of ambiguous patterns , 1990, Biological Cybernetics.

[16]  A. Kristofferson,et al.  The timing of interresponse intervals , 1973 .

[17]  Silke Dodel,et al.  Building neurocognitive networks with a distributed functional architecture. , 2011, Advances in experimental medicine and biology.

[18]  A. Nobre,et al.  Dissociating explicit timing from temporal expectation with fMRI , 2008, Current Opinion in Neurobiology.

[19]  Stefan C. Kremer,et al.  Spatiotemporal Connectionist Networks: A Taxonomy and Review , 2001, Neural Computation.

[20]  D. G. MacKay,et al.  Metaphor and Thought , 1980 .

[21]  Richard Reviewer-Granger Unified Theories of Cognition , 1991, Journal of Cognitive Neuroscience.

[22]  Philipp Slusallek,et al.  Introduction to real-time ray tracing , 2005, SIGGRAPH Courses.

[23]  Ching Y. Suen,et al.  Computer recognition and human production of handwriting , 1989 .

[24]  Viktor K. Jirsa,et al.  Theory of the Relation between Human Brain Activity (MEG) and Hand Movements , 2000, NeuroImage.

[25]  Hans-Leo Teulings,et al.  The Elementary Units of Programming in Handwriting , 1986 .

[26]  Ernst Hairer,et al.  Solving Ordinary Differential Equations I: Nonstiff Problems , 2009 .

[27]  A. Selverston,et al.  Dynamical principles in neuroscience , 2006 .

[28]  D. Poeppel,et al.  Speech perception at the interface of neurobiology and linguistics , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[29]  Tamar Flash,et al.  Motor primitives in vertebrates and invertebrates , 2005, Current Opinion in Neurobiology.

[30]  S. Grossberg Adaptive Resonance Theory , 2006 .

[31]  Auke Jan Ijspeert,et al.  Central pattern generators for locomotion control in animals and robots: A review , 2008, Neural Networks.

[32]  A. Liberman,et al.  On the relation of speech to language , 2000, Trends in Cognitive Sciences.

[33]  Ramón Huerta,et al.  Transient Cognitive Dynamics, Metastability, and Decision Making , 2008, PLoS Comput. Biol..

[34]  J. Kelso Phase transitions and critical behavior in human bimanual coordination. , 1984, The American journal of physiology.

[35]  P. Viviani,et al.  A developmental study of the relationship between geometry and kinematics in drawing movements. , 1991, Journal of experimental psychology. Human perception and performance.

[36]  Viktor K. Jirsa,et al.  Connecting Cortical and Behavioral Dynamics: Bimanual Coordination , 1998, Neural Computation.

[37]  A. Kristofferson,et al.  Response delays and the timing of discrete motor responses , 1973 .

[38]  Professor Dr. Dr. h.c. Hermann Haken,et al.  Synergetic Computers and Cognition , 1991, Springer Series in Synergetics.

[39]  G. Lakoff,et al.  Women, Fire, and Dangerous Things: What Categories Reveal about the Mind , 1988 .

[40]  A J Thomassen,et al.  Low-frequency periodicity in the coordination of progressive handwriting. , 1998, Acta psychologica.

[41]  J A Scott Kelso,et al.  Perturbation-Induced False Starts as a Test of the Jirsa-Kelso Excitator Model , 2009, Journal of motor behavior.

[42]  E. Bizzi,et al.  Linear combinations of primitives in vertebrate motor control. , 1994, Proceedings of the National Academy of Sciences of the United States of America.

[43]  Viktor K. Jirsa,et al.  A theoretical model of phase transitions in the human brain , 1994, Biological Cybernetics.

[44]  Elger L. Abrahamse,et al.  Cognitive Processing in New and Practiced Discrete Keying Sequences , 2010, Front. Psychology.

[45]  William M. Stern,et al.  Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex , 2007, Nature Neuroscience.

[46]  A. Newell Unified Theories of Cognition , 1990 .

[47]  W. Ashby,et al.  Every Good Regulator of a System Must Be a Model of That System , 1970 .

[48]  Hermann Haken Synergetic computers and cognition: a top-down approach to neural nets , 1991 .

[49]  Pier-Giorgio Zanone,et al.  The Coordination Dynamics of Learning: Theoretical Structure and Experimental Agenda , 1994 .

[50]  A. Daffertshofer,et al.  Estimating Kramers–Moyal coefficients in short and non-stationary data sets , 2006 .

[51]  Karl J. Friston,et al.  A Hierarchy of Time-Scales and the Brain , 2008, PLoS Comput. Biol..

[52]  W. Freeman How Brains Make Up Their Minds , 1999 .

[53]  Gerard P. van Galen,et al.  Programming in handwriting: Reaction time and movement time as a function of sequence length , 1983 .

[54]  Karl J. Friston,et al.  Predictive coding under the free-energy principle , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[55]  Viktor K. Jirsa,et al.  Distinct timing mechanisms are implicated in distinct circle drawing tasks , 2010, Neuroscience Letters.

[56]  E Bizzi,et al.  Motor learning through the combination of primitives. , 2000, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[57]  Hans-Leo Teulings,et al.  A DESCRIPTION OF HANDWRITING IN TERMS OF MAIN AXES , 1989 .

[58]  Ajay S. Pillai Structured Flows on Manifolds: Distributed functional architectures , 2008 .

[59]  P. Holmes,et al.  Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields , 1983, Applied Mathematical Sciences.

[60]  George Houghton,et al.  The problem of serial order: a neural network model of sequence learning and recall , 1990 .

[61]  D. Cheyne,et al.  Dynamic cortical activity in the human brain reveals motor equivalence , 1998, Nature.

[62]  William Sulis,et al.  Nonlinear Dynamics in Human Behavior , 1996 .

[63]  H. Haken Synergetics: an Introduction, Nonequilibrium Phase Transitions and Self-organization in Physics, Chemistry, and Biology , 1977 .

[64]  J. Kelso,et al.  The Excitator as a Minimal Model for the Coordination Dynamics of Discrete and Rhythmic Movement Generation , 2005, Journal of motor behavior.

[65]  Jean Petitot,et al.  Morphodynamics and attractor syntax: constituency in visual perception and cognitive grammar , 1996 .

[66]  W B Verwey,et al.  Concatenating familiar movement sequences: the versatile cognitive processor. , 2001, Acta psychologica.

[67]  Ramón Huerta,et al.  Reproducible sequence generation in random neural ensembles. , 2004, Physical review letters.

[68]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[69]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[70]  E. Bizzi,et al.  Article history: , 2005 .

[71]  K Friston,et al.  Stochastic Dynamic Causal Modelling , 2009, NeuroImage.

[72]  Rahul Sarpeshkar,et al.  Ultra Low Power Bioelectronics: Fundamentals, Biomedical Applications, and Bio-Inspired Systems , 2010 .

[73]  G. Lakoff,et al.  Where mathematics comes from : how the embodied mind brings mathematics into being , 2002 .

[74]  G. Lakoff The Contemporary Theory of Metaphor , 1993 .

[75]  A. Thomassen,et al.  Preparation of partly precued handwriting movements: The size of movement units in handwriting , 1983 .

[76]  Robert Kozma,et al.  The KIV model of intentional dynamics and decision making , 2009, Neural Networks.

[77]  S. Grossberg Biological competition: Decision rules, pattern formation, and oscillations. , 1980, Proceedings of the National Academy of Sciences of the United States of America.

[78]  J. Kelso,et al.  From the Cover : The phi complex as a neuromarker of human social coordination , 2007 .

[79]  Gavan Lintern,et al.  Dynamic patterns: The self-organization of brain and behavior , 1997, Complex.

[80]  Frédéric Alexandre,et al.  Connectionist-Symbolic Integration: From Unified to Hybrid Approaches , 1996 .

[81]  H. Maturana,et al.  Autopoiesis and Cognition : The Realization of the Living (Boston Studies in the Philosophy of Scie , 1980 .

[82]  T. Gelder,et al.  Mind as Motion: Explorations in the Dynamics of Cognition , 1995 .

[83]  H. Zelaznik,et al.  The Cerebellum and Event Timing , 2002, Annals of the New York Academy of Sciences.

[84]  Karl J. Friston,et al.  Action and behavior: a free-energy formulation , 2010, Biological Cybernetics.

[85]  Gerard P. van Galen,et al.  Handwriting: Issues for a psychomotor theory ☆ , 1991 .

[86]  Jun Tani,et al.  Emergence of Functional Hierarchy in a Multiple Timescale Neural Network Model: A Humanoid Robot Experiment , 2008, PLoS Comput. Biol..

[87]  A. Opstal Dynamic Patterns: The Self-Organization of Brain and Behavior , 1995 .

[88]  H. Zelaznik,et al.  Disrupted Timing of Discontinuous But Not Continuous Movements by Cerebellar Lesions , 2003, Science.

[89]  G. Lakoff Women, fire, and dangerous things : what categories reveal about the mind , 1989 .

[90]  Rob Sturman,et al.  Phase resetting effects for robust cycles between chaotic sets. , 2003, Chaos.

[91]  Stephen Grossberg,et al.  A neural network model for cursive script production , 1993, Biological Cybernetics.

[92]  Zoubin Ghahramani,et al.  Perspectives and problems in motor learning , 2001, Trends in Cognitive Sciences.

[93]  S. Strogatz Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry and Engineering , 1995 .

[94]  William Bechtel,et al.  Connectionism and the Mind: Parallel Processing, Dynamics, and Evolution in Networks , 2002 .

[95]  Karl J. Friston The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[96]  Mikhail I. Rabinovich,et al.  Dynamical Principles of Emotion-Cognition Interaction: Mathematical Images of Mental Disorders , 2010, PloS one.

[97]  Ray Jackendoff,et al.  Précis of Foundations of language: brain, meaning, grammar, evolution. , 2003, The Behavioral and brain sciences.

[98]  Gábor Orosz,et al.  Dynamics on Networks of Cluster States for Globally Coupled Phase Oscillators , 2007, SIAM J. Appl. Dyn. Syst..

[99]  Timothy D. Lee,et al.  Motor Control and Learning: A Behavioral Emphasis , 1982 .

[100]  H. Kao,et al.  Graphonomics : contemporary research in handwriting , 1986 .

[101]  David M. Jacobs,et al.  An Empirical Illustration and Formalization of the Theory of Direct Learning: The Muscle-Based Perception of Kinetic Properties , 2009 .

[102]  Karl J. Friston,et al.  Recognizing Sequences of Sequences , 2009, PLoS Comput. Biol..

[103]  S. Swinnen Interlimb coordination : neural, dynamical, and cognitive constraints , 1994 .

[104]  J. Kelso,et al.  Coordination Dynamics: Issues and Trends , 2004, Understanding Complex Systems.

[105]  R. Schmidt A schema theory of discrete motor skill learning. , 1975 .

[106]  H. Haken,et al.  Oscillations in the perception of ambiguous patterns a model based on synergetics , 1989, Biological Cybernetics.

[107]  Daniel Bullock,et al.  Competitive Queing for Planning and Serial Performance , 2002 .

[108]  Stefan C. Kremer,et al.  On the computational power of Elman-style recurrent networks , 1995, IEEE Trans. Neural Networks.

[109]  Laure Fernandez,et al.  Fitts' law is not continuous in reciprocal aiming , 2009, Proceedings of the Royal Society B: Biological Sciences.

[110]  P. Bach-y-Rita,et al.  Sensory substitution and the human–machine interface , 2003, Trends in Cognitive Sciences.

[111]  E. Hairer,et al.  Solving ordinary differential equations I (2nd revised. ed.): nonstiff problems , 1993 .

[112]  M. Fayol,et al.  Orthographic vs. phonologic syllables in handwriting production , 2009, Cognition.

[113]  K. Nakayama,et al.  Hidden cognitive states revealed in choice reaching tasks , 2009, Trends in Cognitive Sciences.

[114]  Karl J. Friston Hierarchical Models in the Brain , 2008, PLoS Comput. Biol..

[115]  Stephen Grossberg,et al.  A Theory of Human Memory: Self-Organization and Performance of Sensory-Motor Codes, Maps, and Plans , 1982 .

[116]  C. J. Álvarez,et al.  Syllables as processing units in handwriting production. , 2006, Journal of experimental psychology. Human perception and performance.

[117]  H. Haken,et al.  A theoretical model of phase transitions in human hand movements , 2004, Biological Cybernetics.

[118]  Lambert Schomaker,et al.  Between-letter context effects in handwriting trajectories , 1986 .

[119]  John F. Kolen,et al.  Field Guide to Dynamical Recurrent Networks , 2001 .

[120]  Jerome A. Feldman,et al.  From Molecule to Metaphor - A Neural Theory of Language , 2006 .

[121]  Daniel Bullock,et al.  Neural Dynamics of Learning and Performance of Fixed Sequences: Latency Pattern Reorganizations and the N-STREAMS Model , 2002 .

[122]  H. Maturana,et al.  Autopoiesis and Cognition , 1980 .

[123]  R. Téman,et al.  Integral Manifolds and Inertial Manifolds for Dissipative Partial Differential Equations , 1988 .

[124]  Jun Tani,et al.  How Hierarchical Control Self-organizes in Artificial Adaptive Systems , 2005, Adapt. Behav..

[125]  A. C. Yu,et al.  Temporal Hierarchical Control of Singing in Birds , 1996, Science.

[126]  Karl J. Friston,et al.  Frontiers in Neuroinformatics , 2022 .

[127]  J. Palva,et al.  Phase Synchrony among Neuronal Oscillations in the Human Cortex , 2005, The Journal of Neuroscience.

[128]  Daniel Bullock,et al.  Learning and production of movement sequences: behavioral, neurophysiological, and modeling perspectives. , 2004, Human movement science.

[129]  S. Kandel,et al.  Graphemes as Motor Units in the Acquisition of Writing Skills , 2006 .

[130]  G. Ermentrout Dynamic patterns: The self-organization of brain and behavior , 1997 .

[131]  Viktor K. Jirsa,et al.  Distinct Timing Mechanisms Produce Discrete and Continuous Movements , 2008, PLoS Comput. Biol..

[132]  D. Norris,et al.  The primacy model: a new model of immediate serial recall. , 1998, Psychological review.