The Brain's Router: A Cortical Network Model of Serial Processing in the Primate Brain

The human brain efficiently solves certain operations such as object recognition and categorization through a massively parallel network of dedicated processors. However, human cognition also relies on the ability to perform an arbitrarily large set of tasks by flexibly recombining different processors into a novel chain. This flexibility comes at the cost of a severe slowing down and a seriality of operations (100–500 ms per step). A limit on parallel processing is demonstrated in experimental setups such as the psychological refractory period (PRP) and the attentional blink (AB) in which the processing of an element either significantly delays (PRP) or impedes conscious access (AB) of a second, rapidly presented element. Here we present a spiking-neuron implementation of a cognitive architecture where a large number of local parallel processors assemble together to produce goal-driven behavior. The precise mapping of incoming sensory stimuli onto motor representations relies on a “router” network capable of flexibly interconnecting processors and rapidly changing its configuration from one task to another. Simulations show that, when presented with dual-task stimuli, the network exhibits parallel processing at peripheral sensory levels, a memory buffer capable of keeping the result of sensory processing on hold, and a slow serial performance at the router stage, resulting in a performance bottleneck. The network captures the detailed dynamics of human behavior during dual-task-performance, including both mean RTs and RT distributions, and establishes concrete predictions on neuronal dynamics during dual-task experiments in humans and non-human primates.

[1]  C. W. Telford The refractory phase of voluntary and associative responses , 1931 .

[2]  Emil L. Post Formal Reductions of the General Combinatorial Decision Problem , 1943 .

[3]  M. C. Smith,et al.  Theories of the psychological refractory period. , 1967, Psychological bulletin.

[4]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[5]  G. Claxton Cognitive psychology: New directions , 1980 .

[6]  J. Fuster,et al.  Inferotemporal neurons distinguish and retain behaviorally relevant features of visual stimuli. , 1981, Science.

[7]  M. Posner,et al.  Components of visual orienting , 1984 .

[8]  S. Ullman Visual routines , 1984, Cognition.

[9]  H Pashler,et al.  Processing stages in overlapping tasks: evidence for a central bottleneck. , 1984, Journal of experimental psychology. Human perception and performance.

[10]  S. Wise The primate premotor cortex: past, present, and preparatory. , 1985, Annual review of neuroscience.

[11]  C. Bruce,et al.  Primate frontal eye fields. I. Single neurons discharging before saccades. , 1985, Journal of neurophysiology.

[12]  A. Allport,et al.  Selection for action: Some behavioral and neurophysiological considerations of attention and action , 1987 .

[13]  Allen Newell,et al.  SOAR: An Architecture for General Intelligence , 1987, Artif. Intell..

[14]  Geoffrey E. Hinton,et al.  A Distributed Connectionist Production System , 1988, Cogn. Sci..

[15]  B. Baars A cognitive theory of consciousness , 1988 .

[16]  Richard A. Andersen,et al.  A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons , 1988, Nature.

[17]  H. Heuer,et al.  Perspectives on Perception and Action , 1989 .

[18]  Richard J. Brown Neuropsychology Mental Structure , 1989 .

[19]  J. C. Johnston,et al.  Chronometric Evidence for Central Postponement in Temporally Overlapping Tasks , 2003 .

[20]  G E Alexander,et al.  Neural representations of the target (goal) of visually guided arm movements in three motor areas of the monkey. , 1990, Journal of neurophysiology.

[21]  James L. McClelland,et al.  On the control of automatic processes: a parallel distributed processing account of the Stroop effect. , 1990, Psychological review.

[22]  C. Stevens,et al.  Voltage dependence of NMDA-activated macroscopic conductances predicted by single-channel kinetics , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[23]  Richard Reviewer-Granger Unified Theories of Cognition , 1991, Journal of Cognitive Neuroscience.

[24]  J. Changeux,et al.  The Wisconsin Card Sorting Test: theoretical analysis and modeling in a neuronal network. , 1991, Cerebral cortex.

[25]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[26]  K L Shapiro,et al.  Temporary suppression of visual processing in an RSVP task: an attentional blink? . , 1992, Journal of experimental psychology. Human perception and performance.

[27]  D. V. van Essen,et al.  A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[28]  P. Goldman-Rakic,et al.  Prefrontal neuronal activity in rhesus monkeys performing a delayed anti-saccade task , 1993, Nature.

[29]  A. Burkhalter Development of forward and feedback connections between areas V1 and V2 of human visual cortex. , 1993, Cerebral cortex.

[30]  A. Osman,et al.  The locus of dual-task interference: psychological refractory effects on movement-related brain potentials. , 1993, Journal of experimental psychology. Human perception and performance.

[31]  H. Pashler Dual-task interference in simple tasks: data and theory. , 1994, Psychological bulletin.

[32]  Jun Tanji,et al.  Role for supplementary motor area cells in planning several movements ahead , 1994, Nature.

[33]  H. Pashler,et al.  Visual fields Sequential operation of disconnected cerebral hemispheres in split-brain patients , 2002 .

[34]  M. Posner,et al.  Attentional networks , 1994, Trends in Neurosciences.

[35]  K. Rockland,et al.  Divergent feedback connections from areas V4 and TEO in the macaque , 1994, Visual Neuroscience.

[36]  L F Abbott,et al.  Transfer of coded information from sensory to motor networks , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[37]  M. Potter,et al.  A two-stage model for multiple target detection in rapid serial visual presentation. , 1995, Journal of experimental psychology. Human perception and performance.

[38]  T Shallice,et al.  The domain of supervisory processes and temporal organization of behaviour. , 1996, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[39]  M N Shadlen,et al.  Motion perception: seeing and deciding. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[40]  W Richter,et al.  Limitations of temporal resolution in functional MRI , 1997, Magnetic resonance in medicine.

[41]  R. Desimone,et al.  Neural mechanisms of spatial selective attention in areas V1, V2, and V4 of macaque visual cortex. , 1997, Journal of neurophysiology.

[42]  S Dehaene,et al.  A hierarchical neuronal network for planning behavior. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[43]  D E Kieras,et al.  A computational theory of executive cognitive processes and multiple-task performance: Part 1. Basic mechanisms. , 1997, Psychological review.

[44]  D. Amit,et al.  Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. , 1997, Cerebral cortex.

[45]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[46]  E. Vogel,et al.  Electrophysiological Evidence for a Postperceptual Locus of Suppression during the Attentional Blink Time-based Attention and the Attentional Blink , 1998 .

[47]  J. C. Johnston,et al.  Attentional limitations in dual-task performance. , 1998 .

[48]  Pieter R. Roelfsema,et al.  Object-based attention in the primary visual cortex of the macaque monkey , 1998, Nature.

[49]  A Treisman,et al.  Feature binding, attention and object perception. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[50]  Ravi S. Menon,et al.  Mental chronometry using latency-resolved functional MRI. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[51]  S. Luck Sources of Dual-Task Interference: Evidence From Human Electrophysiology , 1998 .

[52]  E. Miller,et al.  Neural Activity in the Primate Prefrontal Cortex during Associative Learning , 1998, Neuron.

[53]  Earl K. Miller,et al.  Selective representation of relevant information by neurons in the primate prefrontal cortex , 1998, Nature.

[54]  P. Jolicoeur Modulation of the attentional blink by on-line response selection: Evidence from speeded and unspeeded Task1 decisions , 1998, Memory & cognition.

[55]  S. Hillyard,et al.  Event-related brain potentials in the study of visual selective attention. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[56]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[57]  J. Schall,et al.  Neural selection and control of visually guided eye movements. , 1999, Annual review of neuroscience.

[58]  V. Lollo,et al.  The attentional blink with targets in different spatial locations , 1999, Psychonomic bulletin & review.

[59]  K. Arnell,et al.  The attentional blink across stimulus modalities: Evidence for central processing limitations. , 1999 .

[60]  E. Large,et al.  The dynamics of attending: How people track time-varying events. , 1999 .

[61]  Pierre Jolickur Concurrent Response-Selection Demands Modulate the Attentional Blink , 1999 .

[62]  A. Allport,et al.  Task switching and the measurement of “switch costs” , 2000, Psychological research.

[63]  Victor A. F. Lamme,et al.  The implementation of visual routines , 2000, Vision Research.

[64]  V. Lamme,et al.  The distinct modes of vision offered by feedforward and recurrent processing , 2000, Trends in Neurosciences.

[65]  J. Duncan,et al.  Common regions of the human frontal lobe recruited by diverse cognitive demands , 2000, Trends in Neurosciences.

[66]  E. Rolls Functions of the Primate Temporal Lobe Cortical Visual Areas in Invariant Visual Object and Face Recognition , 2000, Neuron.

[67]  Alexandre Pouget,et al.  Computational approaches to sensorimotor transformations , 2000, Nature Neuroscience.

[68]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[69]  J. C. Johnston,et al.  Attention and performance. , 2001, Annual review of psychology.

[70]  J. Ridley Studies of Interference in Serial Verbal Reactions , 2001 .

[71]  P Girard,et al.  Feedback connections act on the early part of the responses in monkey visual cortex. , 2001, Journal of neurophysiology.

[72]  A. Anderson,et al.  Effects of phonological length on the attentional blink for words. , 2001, Journal of experimental psychology. Human perception and performance.

[73]  N. Logothetis,et al.  Neurophysiological investigation of the basis of the fMRI signal , 2001, Nature.

[74]  J. Bullier Integrated model of visual processing , 2001, Brain Research Reviews.

[75]  W. Newsome,et al.  Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. , 2001, Journal of neurophysiology.

[76]  John R. Anderson,et al.  Serial modules in parallel: the psychological refractory period and perfect time-sharing. , 2001, Psychological review.

[77]  David J. Freedman,et al.  Categorical representation of visual stimuli in the primate prefrontal cortex. , 2001, Science.

[78]  K. C. Anderson,et al.  Single neurons in prefrontal cortex encode abstract rules , 2001, Nature.

[79]  S. Dehaene,et al.  Towards a cognitive neuroscience of consciousness: basic evidence and a workspace framework , 2001, Cognition.

[80]  R. Ratcliff A diffusion model account of response time and accuracy in a brightness discrimination task: Fitting real data and failing to fit fake but plausible data , 2002, Psychonomic bulletin & review.

[81]  J. Duncan,et al.  Separate and Shared Sources of Dual-Task Cost in Stimulus Identification and Response Selection , 2002, Cognitive Psychology.

[82]  Xiao-Jing Wang,et al.  Probabilistic Decision Making by Slow Reverberation in Cortical Circuits , 2002, Neuron.

[83]  Bijan Pesaran,et al.  Temporal structure in neuronal activity during working memory in macaque parietal cortex , 2000, Nature Neuroscience.

[84]  M. Shadlen,et al.  Response of Neurons in the Lateral Intraparietal Area during a Combined Visual Discrimination Reaction Time Task , 2002, The Journal of Neuroscience.

[85]  Kin Fai Ellick Wong The Relationship between Attentional Blink and Psychological Refractory Period , 2002 .

[86]  R. Romo,et al.  From sensation to action , 2002, Behavioural Brain Research.

[87]  Jean Bennett,et al.  Lateral Connectivity and Contextual Interactions in Macaque Primary Visual Cortex , 2002, Neuron.

[88]  G. Humphreys,et al.  Attention, spatial representation, and visual neglect: simulating emergent attention and spatial memory in the selective attention for identification model (SAIM). , 2003, Psychological review.

[89]  C. Koch,et al.  Is perception discrete or continuous? , 2003, Trends in Cognitive Sciences.

[90]  E. Koechlin,et al.  The Architecture of Cognitive Control in the Human Prefrontal Cortex , 2003, Science.

[91]  E. Miller,et al.  Neural circuits subserving the retrieval and maintenance of abstract rules. , 2003, Journal of neurophysiology.

[92]  J. Changeux,et al.  A neuronal network model linking subjective reports and objective physiological data during conscious perception , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[93]  Pieter R Roelfsema,et al.  Subtask sequencing in the primary visual cortex , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[94]  A. Graybiel,et al.  Representation of Action Sequence Boundaries by Macaque Prefrontal Cortical Neurons , 2003, Science.

[95]  E. Rolls,et al.  Attention and working memory: a dynamical model of neuronal activity in the prefrontal cortex , 2003, The European journal of neuroscience.

[96]  M. Shadlen,et al.  A role for neural integrators in perceptual decision making. , 2003, Cerebral cortex.

[97]  A. Engel,et al.  Event-related potential correlates of the attentional blink phenomenon. , 2003, Brain research. Cognitive brain research.

[98]  R. Goebel,et al.  Tracking cognitive processes with functional MRI mental chronometry , 2003, Current Opinion in Neurobiology.

[99]  E. Rolls,et al.  A Neurodynamical cortical model of visual attention and invariant object recognition , 2004, Vision Research.

[100]  M. Chun,et al.  The Neural Fate of Consciously Perceived and Missed Events in the Attentional Blink , 2004, Neuron.

[101]  Philip L. Smith,et al.  Psychology and neurobiology of simple decisions , 2004, Trends in Neurosciences.

[102]  Yuhong Jiang,et al.  Resolving dual-task interference: an fMRI study , 2004, NeuroImage.

[103]  Masataka Watanabe Frontal units of the monkey coding the associative significance of visual and auditory stimuli , 2004, Experimental Brain Research.

[104]  Alicia M. Helion,et al.  Dissociating sources of dual-task interference using human electrophysiology , 2004, Psychonomic bulletin & review.

[105]  John R Anderson,et al.  An integrated theory of the mind. , 2004, Psychological review.

[106]  Xiao-Jing Wang,et al.  Effects of Neuromodulation in a Cortical Network Model of Object Working Memory Dominated by Recurrent Inhibition , 2004, Journal of Computational Neuroscience.

[107]  M. Watanabe Prefrontal unit activity during associative learning in the monkey , 2004, Experimental Brain Research.

[108]  N. Kanwisher,et al.  Functional Magnetic Resonance Imaging Provides New Constraints on Theories of the Psychological Refractory Period , 2004, Psychological science.

[109]  Bruno A. Olshausen,et al.  A multiscale dynamic routing circuit for forming size- and position-invariant object representations , 1995, Journal of Computational Neuroscience.

[110]  Emilio Salinas,et al.  Fast Remapping of Sensory Stimuli onto Motor Actions on the Basis of Contextual Modulation , 2004, The Journal of Neuroscience.

[111]  Jonathan D. Cohen,et al.  Prefrontal cortex and flexible cognitive control: rules without symbols. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[112]  M. Sigman,et al.  Opinion TRENDS in Cognitive Sciences Vol.9 No.7 July 2005 The neural code for written words: a proposal , 2022 .

[113]  John J. Foxe,et al.  Multisensory contributions to low-level, ‘unisensory’ processing , 2005, Current Opinion in Neurobiology.

[114]  R. Marois,et al.  Capacity limits of information processing in the brain , 2005, Trends in Cognitive Sciences.

[115]  E. Rolls,et al.  Synaptic and spiking dynamics underlying reward reversal in the orbitofrontal cortex. , 2004, Cerebral cortex.

[116]  S. Dehaene,et al.  Timing of the brain events underlying access to consciousness during the attentional blink , 2005, Nature Neuroscience.

[117]  P. Roelfsema Elemental operations in vision , 2005, Trends in Cognitive Sciences.

[118]  J. Enns,et al.  The attentional blink: Resource depletion or temporary loss of control? , 2005, Psychological research.

[119]  M. Sigman,et al.  Parsing a Cognitive Task: A Characterization of the Mind's Bottleneck , 2005, PLoS biology.

[120]  Dietmar Heinke,et al.  Selective Attention for Identification Model: Simulating visual neglect , 2005, Comput. Vis. Image Underst..

[121]  E. Rolls,et al.  Attention, short-term memory, and action selection: A unifying theory , 2005, Progress in Neurobiology.

[122]  P. Jolicoeur,et al.  Central processing overlap modulates P3 latency , 2005, Experimental Brain Research.

[123]  John G. Taylor,et al.  A neurodynamic model of the attentional blink. , 2005, Brain research. Cognitive brain research.

[124]  Mark S. Gilzenrat,et al.  The role of the locus coeruleus in mediating the attentional blink: a neurocomputational theory. , 2005, Journal of experimental psychology. General.

[125]  R. O’Reilly Biologically Based Computational Models of High-Level Cognition , 2006, Science.

[126]  David J. Freedman,et al.  Experience-dependent representation of visual categories in parietal cortex , 2006, Nature.

[127]  M. Nieuwenstein Top-down controlled, delayed selection in the attentional blink. , 2006, Journal of experimental psychology. Human perception and performance.

[128]  P. Roelfsema Cortical algorithms for perceptual grouping. , 2006, Annual review of neuroscience.

[129]  Xiao-Jing Wang,et al.  A Recurrent Network Mechanism of Time Integration in Perceptual Decisions , 2006, The Journal of Neuroscience.

[130]  E. Koechlin,et al.  Broca's Area and the Hierarchical Organization of Human Behavior , 2006, Neuron.

[131]  Puiu F. Balan,et al.  Integration of Visuospatial and Effector Information during Symbolically Cued Limb Movements in Monkey Lateral Intraparietal Area , 2006, The Journal of Neuroscience.

[132]  J. Enns,et al.  The attentional blink is not a unitary phenomenon , 2006, Psychological research.

[133]  Christopher L. Asplund,et al.  Isolation of a Central Bottleneck of Information Processing with Time-Resolved fMRI , 2006, Neuron.

[134]  C. Gilbert,et al.  Contour Saliency in Primary Visual Cortex , 2006, Neuron.

[135]  B. Hommel,et al.  Resource sharing in the attentional blink , 2006, Neuroreport.

[136]  M. Sigman,et al.  Dynamics of the Central Bottleneck: Dual-Task and Task Uncertainty , 2006, PLoS biology.

[137]  Xiao-Jing Wang,et al.  Cortico–basal ganglia circuit mechanism for a decision threshold in reaction time tasks , 2006, Nature Neuroscience.

[138]  J. Hulleman,et al.  Spreading the sparing: against a limited-capacity account of the attentional blink , 2007, Psychological research.

[139]  E. Koechlin,et al.  Serial Organization of Human Behavior in the Inferior Parietal Cortex , 2007, The Journal of Neuroscience.

[140]  Gustavo Deco,et al.  The neuronal dynamics underlying cognitive flexibility in set shifting tasks , 2007, Journal of Computational Neuroscience.

[141]  Mariano Sigman,et al.  Hierarchical Coding of Letter Strings in the Ventral Stream: Dissecting the Inner Organization of the Visual Word-Form System , 2007, Neuron.

[142]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[143]  Alexander Maye,et al.  Temporal dynamics of access to consciousness in the attentional blink , 2007, NeuroImage.

[144]  B. Wyble,et al.  The simultaneous type, serial token model of temporal attention and working memory. , 2007, Psychological review.

[145]  Benoit Brisson,et al.  Electrophysiological evidence of central interference in the control of visuospatial attention , 2007, Psychonomic bulletin & review.

[146]  S. Dehaene,et al.  Brain Dynamics Underlying the Nonlinear Threshold for Access to Consciousness , 2007, PLoS biology.

[147]  E. Miller,et al.  A Neural Circuit Model of Flexible Sensorimotor Mapping: Learning and Forgetting on Multiple Timescales , 2007, Neuron.

[148]  R. Verleger,et al.  P3 latency shifts in the attentional blink: Further evidence for second target processing postponement , 2007, Brain Research.

[149]  C. Schroeder,et al.  Neuronal Oscillations and Multisensory Interaction in Primary Auditory Cortex , 2007, Neuron.

[150]  G. Karmos,et al.  Entrainment of Neuronal Oscillations as a Mechanism of Attentional Selection , 2008, Science.

[151]  M. Sigman,et al.  The dynamics of sensory buffers: geometric, spatial, and experience-dependent shaping of iconic memory. , 2008, Journal of vision.

[152]  Mariano Sigman,et al.  Delays without Mistakes: Response Time and Error Distributions in Dual-Task , 2008, PloS one.

[153]  M. Sigman,et al.  Brain Mechanisms of Serial and Parallel Processing during Dual-Task Performance , 2008, The Journal of Neuroscience.

[154]  C. Schroeder,et al.  Neuronal Mechanisms of Cortical Alpha Oscillations in Awake-Behaving Macaques , 2008, The Journal of Neuroscience.

[155]  C. Olivers,et al.  A boost and bounce theory of temporal attention. , 2008, Psychological review.

[156]  John R. Anderson,et al.  SAL: an explicitly pluralistic cognitive architecture , 2008, J. Exp. Theor. Artif. Intell..

[157]  S. Shih The attention cascade model and attentional blink , 2008, Cognitive Psychology.

[158]  M. D’Esposito Working memory. , 2008, Handbook of clinical neurology.

[159]  John Duncan,et al.  Hierarchical coding for sequential task events in the monkey prefrontal cortex , 2008, Proceedings of the National Academy of Sciences.

[160]  B. Wyble,et al.  A reciprocal relationship between bottom-up trace strength and the attentional blink bottleneck: Relating the LC–NE and ST2 models , 2008, Brain Research.

[161]  M. Potter,et al.  Unmasking the attentional blink. , 2009, Journal of experimental psychology. Human perception and performance.

[162]  C. Schroeder,et al.  Low-frequency neuronal oscillations as instruments of sensory selection , 2009, Trends in Neurosciences.

[163]  R. Marois,et al.  The attentional blink: A review of data and theory , 2009, Attention, perception & psychophysics.

[164]  Mariano Sigman,et al.  Frontiers in Computational Neuroscience Computational Neuroscience Neurophysiological Bases of Exponential Sensory Decay and Top-down Memory Retrieval: a Model , 2022 .

[165]  R. VanRullen,et al.  The Phase of Ongoing EEG Oscillations Predicts Visual Perception , 2009, The Journal of Neuroscience.

[166]  M. Sigman,et al.  The Spatial and Temporal Construction of Confidence in the Visual Scene , 2009, PloS one.

[167]  N. Logothetis,et al.  Frequency-Band Coupling in Surface EEG Reflects Spiking Activity in Monkey Visual Cortex , 2009, Neuron.

[168]  P. Jolicoeur,et al.  Reevaluating encoding-capacity limitations as a cause of the attentional blink. , 2009, Journal of experimental psychology. Human perception and performance.

[169]  M. Nieuwenstein,et al.  The attentional blink provides episodic distinctiveness: sparing at a cost. , 2009, Journal of experimental psychology. Human perception and performance.

[170]  E. Koechlin,et al.  Motivation and cognitive control in the human prefrontal cortex , 2009, Nature Neuroscience.

[171]  E. Mavritsaki,et al.  The selective attention for identification model (SAIM): A framework for closing the gap between the behavioural and neurological levels , 2009 .

[172]  Howard Bowman,et al.  A neural network account of binding discrete items into working memory using a distributed pool of flexible resources , 2010 .