Hierarchical Rule Switching in Prefrontal Cortex

Most real-world decision-making problems involve consideration of numerous possible actions, and it is often impossible to evaluate all of them before settling on preferred strategy. In such situations, humans might explore actions more efficiently by searching only the most likely subspace of the whole action space. To study how the brain solves such action selection problems, we designed a Multi Feature Sorting Task in which the task rules defining an optimal action have a hierarchical structure and studied concurrent brain activity using it. The task consisted of two kinds of rule switches: a higher-order switch to search for a rule across different subspaces and a lower-order switch to change a rule within the same subspace. The results revealed that the left dorsolateral prefrontal cortex (DLPFC) was more active in the higher-order switching, and the right fronto-polar cortex (FPC) was significantly activated with the lower-order switching. We discuss a possible functional model in the prefrontal cortex where the left DLPFC encodes the hierarchical organization of behaviours and the right FPC maintains and updates multiple behavioural. This interpretation is highly consistent with the previous findings and current theories of hierarchical organization in the prefrontal functional network.

[1]  Allen Newell,et al.  GPS, a program that simulates human thought , 1995 .

[2]  Gereon R. Fink,et al.  Using fMRI to decompose the neural processes underlying the Wisconsin Card Sorting Test , 2006, NeuroImage.

[3]  M. Brodeur,et al.  Prefrontal cortex contribution to associative recognition memory in humans: an event-related functional magnetic resonance imaging study , 2003, Neuroscience Letters.

[4]  F. Rösler,et al.  Brain Activation Modulated by the Comprehension of Normal and Pseudo-word Sentences of Different Processing Demands: A Functional Magnetic Resonance Imaging Study , 2002, NeuroImage.

[5]  E. Kaan,et al.  The brain circuitry of syntactic comprehension , 2002, Trends in Cognitive Sciences.

[6]  M. Botvinick Hierarchical models of behavior and prefrontal function , 2008, Trends in Cognitive Sciences.

[7]  K. Berman,et al.  Fractionating the neural substrate of cognitive control processes , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[8]  S. Joordens,et al.  Investigating a memory-based account of negative priming: support for selection-feature mismatch. , 2000, Journal of experimental psychology. Human perception and performance.

[9]  S. Ishii,et al.  Resolution of Uncertainty in Prefrontal Cortex , 2006, Neuron.

[10]  A. Dove,et al.  Prefrontal cortex activation in task switching: an event-related fMRI study. , 2000, Brain research. Cognitive brain research.

[11]  E. Feigenbaum,et al.  Computers and Thought , 1963 .

[12]  J. Grafman,et al.  Human prefrontal cortex: processing and representational perspectives , 2003, Nature Reviews Neuroscience.

[13]  B. Milner Effects of Different Brain Lesions on Card Sorting: The Role of the Frontal Lobes , 1963 .

[14]  R. Passingham,et al.  The prefrontal cortex: response selection or maintenance within working memory? , 2000, 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002..

[15]  D. A. Grant,et al.  A behavioral analysis of degree of reinforcement and ease of shifting to new responses in a Weigl-type card-sorting problem. , 1948, Journal of experimental psychology.

[16]  Eliot Hazeltine,et al.  Dissociable Contributions of Prefrontal and Parietal Cortices to Response Selection , 2002, NeuroImage.

[17]  A. Owen,et al.  Fractionating attentional control using event-related fMRI. , 2005, Cerebral cortex.

[18]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[19]  Richard Coppola,et al.  Physiological activation of a cortical network during performance of the Wisconsin Card Sorting Test: A positron emission tomography study , 1995, Neuropsychologia.

[20]  Gordon D. Logan,et al.  Hierarchical control of cognitive processes: switching tasks in sequences. , 2006 .

[21]  T. Braver,et al.  Anterior cingulate cortex and response conflict: effects of frequency, inhibition and errors. , 2001, Cerebral cortex.

[22]  C. Rorden,et al.  Cognitive Control Mechanisms Revealed by ERP and fMRI: Evidence from Repeated Task-Switching , 2003, Journal of Cognitive Neuroscience.

[23]  M. Raichle,et al.  Searching for a baseline: Functional imaging and the resting human brain , 2001, Nature Reviews Neuroscience.

[24]  E. Koechlin,et al.  Anterior Prefrontal Function and the Limits of Human Decision-Making , 2007, Science.

[25]  A. Allport,et al.  Task switching and the measurement of “switch costs” , 2000, Psychological research.

[26]  M. Rugg,et al.  Retrieval processing and episodic memory , 2000, Trends in Cognitive Sciences.

[27]  E. Koechlin,et al.  The Architecture of Cognitive Control in the Human Prefrontal Cortex , 2003, Science.

[28]  E. Koechlin,et al.  Broca's Area and the Hierarchical Organization of Human Behavior , 2006, Neuron.

[29]  John R. Anderson,et al.  The role of prefrontal cortex and posterior parietal cortex in task switching. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[30]  R Kakigi,et al.  Neural activities during Wisconsin Card Sorting Test--MEG observation. , 2001, Brain research. Cognitive brain research.

[31]  Anthony D Wagner,et al.  Executive Control during Episodic Retrieval Multiple Prefrontal Processes Subserve Source Memory , 2002, Neuron.

[32]  Randi C. Martin,et al.  Language processing: functional organization and neuroanatomical basis. , 2003, Annual review of psychology.

[33]  M. Botvinick,et al.  Anterior cingulate cortex, error detection, and the online monitoring of performance. , 1998, Science.

[34]  S. Thompson-Schill,et al.  The frontal lobes and the regulation of mental activity , 2005, Current Opinion in Neurobiology.

[35]  M. D’Esposito,et al.  Modulation of task-related neural activity in task-switching: an fMRI study. , 2000, Brain research. Cognitive brain research.

[36]  D. Plaut,et al.  Doing without schema hierarchies: a recurrent connectionist approach to normal and impaired routine sequential action. , 2004, Psychological review.

[37]  Timothy Edward John Behrens,et al.  How Green Is the Grass on the Other Side? Frontopolar Cortex and the Evidence in Favor of Alternative Courses of Action , 2009, Neuron.

[38]  Y. Miyashita,et al.  Transient activation of inferior prefrontal cortex during cognitive set shifting , 1998, Nature Neuroscience.

[39]  C. Liston,et al.  Anterior Cingulate and Posterior Parietal Cortices Are Sensitive to Dissociable Forms of Conflict in a Task-Switching Paradigm , 2006, Neuron.

[40]  Jonathan D. Cohen,et al.  Conflict monitoring versus selection-for-action in anterior cingulate cortex , 1999, Nature.

[41]  Jeremy R. Reynolds,et al.  Neural Mechanisms of Transient and Sustained Cognitive Control during Task Switching , 2003, Neuron.

[42]  Thomas E. Hazy,et al.  Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[43]  Morris Moscovitch,et al.  Strategic retrieval and the frontal lobes: Evidence from confabulation and amnesia , 1997, Neuropsychologia.

[44]  Karl J. Friston,et al.  Anterior prefrontal cortex mediates rule learning in humans. , 2001, Cerebral cortex.

[45]  C. Büchel,et al.  Broca's area and the language instinct , 2003, Nature Neuroscience.

[46]  M. Petrides,et al.  Wisconsin Card Sorting Revisited: Distinct Neural Circuits Participating in Different Stages of the Task Identified by Event-Related Functional Magnetic Resonance Imaging , 2001, The Journal of Neuroscience.

[47]  Sridhar Mahadevan,et al.  Recent Advances in Hierarchical Reinforcement Learning , 2003, Discret. Event Dyn. Syst..

[48]  Jonathan D. Cohen,et al.  Between-Task Competition and Cognitive Control in Task Switching , 2006, The Journal of Neuroscience.

[49]  Jonathan D. Cohen,et al.  The Impact of Cognitive Deficits on Conflict Monitoring , 2006, Psychological science.

[50]  B. Richmond,et al.  Anterior Cingulate: Single Neuronal Signals Related to Degree of Reward Expectancy , 2002, Science.

[51]  Timothy E. J. Behrens,et al.  Choice, uncertainty and value in prefrontal and cingulate cortex , 2008, Nature Neuroscience.

[52]  David Badre,et al.  Cognitive control, hierarchy, and the rostro–caudal organization of the frontal lobes , 2008, Trends in Cognitive Sciences.

[53]  K. Zilles,et al.  Differential Involvement of Parietal and Precentral Regions in Movement Preparation and Motor Intention , 2002, The Journal of Neuroscience.

[54]  Jürgen Schmidhuber,et al.  HQ-Learning , 1997, Adapt. Behav..

[55]  P W Burgess,et al.  Confabulation and the control of recollection. , 1996, Memory.

[56]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[57]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[58]  Volkmar Glauche,et al.  The human action recognition system and its relationship to Broca’s area: an fMRI study , 2003, NeuroImage.

[59]  E. Procyk,et al.  Brain activity during observation of actions. Influence of action content and subject's strategy. , 1997, Brain : a journal of neurology.

[60]  J. Tanji,et al.  Role for cingulate motor area cells in voluntary movement selection based on reward. , 1998, Science.

[61]  T. Goldberg,et al.  Uncoupling Cognitive Workload and Prefrontal Cortical Physiology: A PET rCBF Study , 1998, NeuroImage.

[62]  Brian Knutson,et al.  Dissociation of reward anticipation and outcome with event-related fMRI , 2001, Neuroreport.

[63]  Y. Miyashita,et al.  Hemispheric asymmetry in human lateral prefrontal cortex during cognitive set shifting , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[64]  J. Tanji,et al.  Categorization of behavioural sequences in the prefrontal cortex , 2007, Nature.

[65]  A. Dale,et al.  Dorsal anterior cingulate cortex: A role in reward-based decision making , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[66]  D. Yves von Cramon,et al.  Neuroimaging of Performance Monitoring: Error Detection and Beyond , 2004, Cortex.