Necessary Contributions of Human Frontal Lobe Subregions to Reward Learning in a Dynamic, Multidimensional Environment

Real-world decisions are typically made between options that vary along multiple dimensions, requiring prioritization of the important dimensions to support optimal choice. Learning in this setting depends on attributing decision outcomes to the dimensions with predictive relevance rather than to dimensions that are irrelevant and nonpredictive. This attribution problem is computationally challenging, and likely requires an interplay between selective attention and reward learning. Both these processes have been separately linked to the prefrontal cortex, but little is known about how they combine to support learning the reward value of multidimensional stimuli. Here, we examined the necessary contributions of frontal lobe subregions in attributing feedback to relevant and irrelevant dimensions on a trial-by-trial basis in humans. Patients with focal frontal lobe damage completed a demanding reward learning task where options varied on three dimensions, only one of which predicted reward. Participants with left lateral frontal lobe damage attributed rewards to irrelevant dimensions, rather than the relevant dimension. Damage to the ventromedial frontal lobe also impaired learning about the relevant dimension, but did not increase reward attribution to irrelevant dimensions. The results argue for distinct roles for these two regions in learning the value of multidimensional decision options under dynamic conditions, with the lateral frontal lobe required for selecting the relevant dimension to associate with reward, and the ventromedial frontal lobe required to learn the reward association itself. SIGNIFICANCE STATEMENT The real world is complex and multidimensional; how do we attribute rewards to predictive features when surrounded by competing cues? Here, we tested the critical involvement of human frontal lobe subregions in a probabilistic, multidimensional learning environment, asking whether focal lesions affected trial-by-trial attribution of feedback to relevant and irrelevant dimensions. The left lateral frontal lobe was required for filtering option dimensions to allow appropriate feedback attribution, while the ventromedial frontal lobe was necessary for learning the value of features in the relevant dimension. These findings argue that selective attention and associative learning processes mediated by anatomically distinct frontal lobe subregions are both critical for adaptive choice in more complex, ecologically valid settings.

[1]  Lesley K. Fellows,et al.  Are core component processes of executive function dissociable within the frontal lobes? Evidence from humans with focal prefrontal damage , 2013, Cortex.

[2]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[3]  Lesley K Fellows,et al.  Ventromedial Frontal Cortex Is Critical for Guiding Attention to Reward-Predictive Visual Features in Humans , 2015, The Journal of Neuroscience.

[4]  J. Kruschke Attention in Learning , 2003 .

[5]  Timothy Edward John Behrens,et al.  Ventromedial Prefrontal and Anterior Cingulate Cortex Adopt Choice and Default Reference Frames during Sequential Multi-Alternative Choice , 2013, The Journal of Neuroscience.

[6]  E. Brunner,et al.  The Nonparametric Behrens‐Fisher Problem: Asymptotic Theory and a Small‐Sample Approximation , 2000 .

[7]  Robert Desimone,et al.  Top–Down Attentional Deficits in Macaques with Lesions of Lateral Prefrontal Cortex , 2007, The Journal of Neuroscience.

[8]  M. Posner,et al.  Orienting of Attention* , 1980, The Quarterly journal of experimental psychology.

[9]  N. Mackintosh,et al.  Two theories of attention: a review and a possible integration , 2010 .

[10]  S. Thompson-Schill,et al.  A matched filter hypothesis for cognitive control , 2014, Neuropsychologia.

[11]  Nancy Kanwisher,et al.  fMRI evidence for objects as the units of attentional selection , 1999, Nature.

[12]  E. Rolls,et al.  Reward-related Reversal Learning after Surgical Excisions in Orbito-frontal or Dorsolateral Prefrontal Cortex in Humans , 2004, Journal of Cognitive Neuroscience.

[13]  Robert C. Wilson,et al.  Inferring Relevance in a Changing World , 2012, Front. Hum. Neurosci..

[14]  Nils Kolling,et al.  A neural mechanism underlying failure of optimal choice with multiple alternatives , 2014, Nature Neuroscience.

[15]  M. Corballis,et al.  Winding one's ps and qs: mental rotation and mirror-image discrimination. , 1984, Journal of experimental psychology. Human perception and performance.

[16]  Timothy Edward John Behrens,et al.  Separable Learning Systems in the Macaque Brain and the Role of Orbitofrontal Cortex in Contingent Learning , 2010, Neuron.

[17]  Timothy E. J. Behrens,et al.  Hierarchical competitions subserving multi-attribute choice , 2014, Nature Neuroscience.

[18]  Yadin Dudai,et al.  The posterior parietal cortex in recognition memory: A neuropsychological study , 2008, Neuropsychologia.

[19]  L. Chelazzi,et al.  Behavioral/systems/cognitive Reward Changes Salience in Human Vision via the Anterior Cingulate , 2022 .

[20]  B. Milner Effects of Different Brain Lesions on Card Sorting: The Role of the Frontal Lobes , 1963 .

[21]  Robert C. Wilson,et al.  Reinforcement Learning in Multidimensional Environments Relies on Attention Mechanisms , 2015, The Journal of Neuroscience.

[22]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[23]  Timothy E. J. Behrens,et al.  Frontal Cortex Subregions Play Distinct Roles in Choices between Actions and Stimuli , 2008, The Journal of Neuroscience.

[24]  N. Mackintosh A Theory of Attention: Variations in the Associability of Stimuli with Reinforcement , 1975 .

[25]  Timothy Edward John Behrens,et al.  Reward-Guided Learning with and without Causal Attribution , 2016, Neuron.

[26]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[27]  E. Renzi,et al.  The token test: A sensitive test to detect receptive disturbances in aphasics. , 1962, Brain : a journal of neurology.

[28]  K. Boone,et al.  Handbook of Normative Data for Neuropsychological Assessment , 1999 .

[29]  Jon Driver,et al.  Repetition streaks increase perceptual sensitivity in visual search of brief displays , 2008, Visual cognition.

[30]  L. Cooper Mental rotation of random two-dimensional shapes , 1975, Cognitive Psychology.

[31]  J. Gottlieb,et al.  Distinct neural mechanisms of distractor suppression in the frontal and parietal lobe , 2012, Nature Neuroscience.

[32]  Edward T. Bullmore,et al.  The role of the orbitofrontal cortex in human discrimination learning , 2008, Neuropsychologia.

[33]  Antonio Rangel,et al.  Stimulus Value Signals in Ventromedial PFC Reflect the Integration of Attribute Value Signals Computed in Fusiform Gyrus and Posterior Superior Temporal Gyrus , 2013, The Journal of Neuroscience.

[34]  Jon Driver,et al.  Fortune and reversals of fortune in visual search: Reward contingencies for pop-out targets affect search efficiency and target repetition effects , 2010, Attention, perception & psychophysics.

[35]  Stanislas Dehaene,et al.  Breaking the symmetry: Mirror discrimination for single letters but not for pictures in the Visual Word Form Area , 2011, NeuroImage.

[36]  Y. Nakajima,et al.  Mirror-image discrimination in the literate brain: a causal role for the left occpitotemporal cortex , 2014, Front. Psychol..

[37]  G. Schott,et al.  Mirror writing: neurological reflections on an unusual phenomenon , 2006, Journal of Neurology, Neurosurgery & Psychiatry.

[38]  E. Perret The left frontal lobe of man and the suppression of habitual responses in verbal categorical behaviour. , 1974, Neuropsychologia.

[39]  D. Gaffan,et al.  Asymmetry of attentional set in rhesus monkeys learning colour and shape discriminations , 2006, Quarterly journal of experimental psychology.

[40]  J. Cummings,et al.  The Montreal Cognitive Assessment, MoCA: A Brief Screening Tool For Mild Cognitive Impairment , 2005, Journal of the American Geriatrics Society.

[41]  Anjan Chatterjee,et al.  Patient Registries in Cognitive Neuroscience Research: Advantages, Challenges, and Practical Advice , 2008, Journal of Cognitive Neuroscience.

[42]  Bolton K. H. Chau,et al.  Prioritising the relevant information for learning and decision making within orbital and ventromedial prefrontal cortex , 2015, Current Opinion in Behavioral Sciences.

[43]  L. Fellows,et al.  Material-specific interference control is dissociable and lateralized in human prefrontal cortex , 2014, Neuropsychologia.

[44]  M. Husain,et al.  Control over Conflict during Movement Preparation: Role of Posterior Parietal Cortex , 2008, Neuron.

[45]  A. Hillis,et al.  Dissociation between egocentric and allocentric visuospatial and tactile neglect in acute stroke , 2008, Cortex.

[46]  R. Saunders,et al.  Prefrontal mechanisms of behavioral flexibility, emotion regulation and value updating , 2013, Nature Neuroscience.

[47]  Pieter R. Roelfsema,et al.  A learning rule that explains how rewards teach attention , 2015 .

[48]  L. Fellows,et al.  Double Dissociation of Stimulus-Value and Action-Value Learning in Humans with Orbitofrontal or Anterior Cingulate Cortex Damage , 2011, The Journal of Neuroscience.

[49]  Matthew F.S. Rushworth,et al.  Contrasting Roles for Orbitofrontal Cortex and Amygdala in Credit Assignment and Learning in Macaques , 2015, Neuron.

[50]  T. Shallice,et al.  Multiple frontal systems controlling response speed , 2005, Neuropsychologia.

[51]  J. Gläscher,et al.  Lesion mapping of cognitive control and value-based decision making in the prefrontal cortex , 2012, Proceedings of the National Academy of Sciences.

[52]  G. Bower,et al.  Depth of processing pictures of faces and recognition memory , 1974 .

[53]  Sébastien Tremblay,et al.  Attentional Filtering of Visual Information by Neuronal Ensembles in the Primate Lateral Prefrontal Cortex , 2015, Neuron.

[54]  J. Malmaud,et al.  Focusing Attention on the Health Aspects of Foods Changes Value Signals in vmPFC and Improves Dietary Choice , 2011, The Journal of Neuroscience.

[55]  J. Pearce,et al.  A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980, Psychological review.

[56]  L. Fellows,et al.  Beyond Reversal: A Critical Role for Human Orbitofrontal Cortex in Flexible Learning from Probabilistic Feedback , 2010, The Journal of Neuroscience.

[57]  K. Nakayama,et al.  Priming of pop-out: I. Role of features , 1994, Memory & cognition.

[58]  Daniel Y. Kimberg,et al.  Power in Voxel-based Lesion-Symptom Mapping , 2007, Journal of Cognitive Neuroscience.

[59]  Jan Theeuwes,et al.  Reward Guides Vision when It's Your Thing: Trait Reward-Seeking in Reward-Mediated Visual Priming , 2010, PloS one.

[60]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[61]  N. Mackintosh Overshadowing and stimulus intensity , 1976, Animal learning & behavior.

[62]  G. Campana,et al.  Where perception meets memory: A review of repetition priming in visual search tasks , 2010, Attention, perception & psychophysics.

[63]  Joshua W. Brown,et al.  Neural Mechanisms of Credit Assignment in a Multicue Environment , 2016, The Journal of Neuroscience.

[64]  Chris Rorden,et al.  Lesion Mapping of Cognitive Abilities Linked to Intelligence , 2009, Neuron.

[65]  N. Jewell,et al.  To GEE or Not to GEE: Comparing Population Average and Mixed Models for Estimating the Associations Between Neighborhood Risk Factors and Health , 2010, Epidemiology.

[66]  E. Miller,et al.  Top-Down Versus Bottom-Up Control of Attention in the Prefrontal and Posterior Parietal Cortices , 2007, Science.

[67]  T. Robbins,et al.  Dissociation in prefrontal cortex of affective and attentional shifts , 1996, Nature.

[68]  M. Farah,et al.  Ventromedial frontal cortex mediates affective shifting in humans: evidence from a reversal learning paradigm. , 2003, Brain : a journal of neurology.

[69]  Charles M. Butter,et al.  Perseveration in extinction and in discrimination reversal tasks following selective frontal ablations in Macaca mulatta , 1969 .