Learning from feedback: The neural mechanisms of feedback processing facilitating better performance

Different levels of feedback, from sensory signals to verbal advice, are needed not only for learning new skills, but also for monitoring performance. A great deal of research has focused on the electrophysiological correlates of feedback processing and how they relate to good learning. In this paper, studies on the EEG correlates of learning from feedback are reviewed. The main objective is to discuss these findings whilst also considering some key theoretical aspects of learning. The learning processes, its operational definition and the feedback characteristics are discussed and used as reference for integrating the findings in the literature. The EEG correlates of feedback processing for learning using various analytical approaches are discussed, including ERPs, oscillations and inter-site synchronization. How these EEG responses to feedback are related to learning is discussed, highlighting the gaps in the literature and suggesting future directions for understanding the neural underpinnings of learning from feedback.

[1]  D. Wolpert,et al.  Principles of sensorimotor learning , 2011, Nature Reviews Neuroscience.

[2]  D. Tucker,et al.  Electrophysiological Responses to Errors and Feedback in the Process of Action Regulation , 2003, Psychological science.

[3]  Marco Rotonda,et al.  EEG oscillatory activity associated to monetary gain and loss signals in a learning task: effects of attentional impulsivity and learning ability. , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[4]  A. Graybiel,et al.  Synchronous, Focally Modulated β-Band Oscillations Characterize Local Field Potential Activity in the Striatum of Awake Behaving Monkeys , 2003, The Journal of Neuroscience.

[5]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[6]  Joydeep Bhattacharya,et al.  Processing Graded Feedback: Electrophysiological Correlates of Learning from Small and Large Errors , 2014, Journal of Cognitive Neuroscience.

[7]  Rich S. W. Masters,et al.  Marginally perceptible outcome feedback, motor learning and implicit processes , 2009, Consciousness and Cognition.

[8]  W. Schultz Predictive reward signal of dopamine neurons. , 1998, Journal of neurophysiology.

[9]  A. Rodríguez-Fornells,et al.  Brain potentials related to self-generated and external information used for performance monitoring , 2005, Clinical Neurophysiology.

[10]  Michael X. Cohen,et al.  Frontal theta reflects uncertainty and unexpectedness during exploration and exploitation. , 2012, Cerebral cortex.

[11]  P. Celnik,et al.  Dissociating the roles of the cerebellum and motor cortex during adaptive learning: the motor cortex retains what the cerebellum learns. , 2011, Cerebral cortex.

[12]  Michael X. Cohen,et al.  Cortical electrophysiological network dynamics of feedback learning , 2011, Trends in Cognitive Sciences.

[13]  Clay B. Holroyd,et al.  Reinforcement-related brain potentials from medial frontal cortex: origins and functional significance , 2004, Neuroscience & Biobehavioral Reviews.

[14]  Tomifusa Kuboki,et al.  Error-related negativity reflects detection of negative reward prediction error , 2004, Neuroreport.

[15]  Hauke R. Heekeren,et al.  Temporal dynamics of prediction error processing during reward-based decision making , 2010, NeuroImage.

[16]  Maarten A. S. Boksem,et al.  The Importance of Failure: Feedback Related Negativity Predicts Motor Learning Efficiency , 2009, NeuroImage.

[17]  D. Meyer,et al.  A Neural System for Error Detection and Compensation , 1993 .

[18]  James W. Tanaka,et al.  Learning to Become an Expert: Reinforcement Learning and the Acquisition of Perceptual Expertise , 2009, Journal of Cognitive Neuroscience.

[19]  I. Daum,et al.  Learning‐related changes in reward expectancy are reflected in the feedback‐related negativity , 2008, The European journal of neuroscience.

[20]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[21]  Michael Falkenstein,et al.  The use of the subtraction technique in the psychophysiology of response inhibition and conflict , 2004 .

[22]  Josep Marco-Pallarés,et al.  The role of beta-gamma oscillations in unexpected rewards processing , 2012, NeuroImage.

[23]  John J. B. Allen,et al.  Theta lingua franca: a common mid-frontal substrate for action monitoring processes. , 2012, Psychophysiology.

[24]  A. Graybiel,et al.  Highly restricted origin of prefrontal cortical inputs to striosomes in the macaque monkey , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[25]  I. Daum,et al.  It is less than you expected: The feedback-related negativity reflects violations of reward magnitude expectations , 2010, Neuropsychologia.

[26]  Uta Sailer,et al.  Manipulation of feedback expectancy and valence induces negative and positive reward prediction error signals manifest in event-related brain potentials. , 2011, Psychophysiology.

[27]  James F. Cavanagh,et al.  Frontal theta links prediction errors to behavioral adaptation in reinforcement learning , 2010, NeuroImage.

[28]  Michael X. Cohen,et al.  Behavioral / Systems / Cognitive Reinforcement Learning Signals Predict Future Decisions , 2007 .

[29]  T. Paus Primate anterior cingulate cortex: Where motor control, drive and cognition interface , 2001, Nature Reviews Neuroscience.

[30]  John R. Anderson,et al.  Learning from experience: Event-related potential correlates of reward processing, neural adaptation, and behavioral choice , 2012, Neuroscience & Biobehavioral Reviews.

[31]  Peter Ullsperger,et al.  Dissociable medial frontal negativities from a common monitoring system for self- and externally caused failure of goal achievement , 2009, NeuroImage.

[32]  Markus Ullsperger,et al.  Neuropharmacology of performance monitoring , 2009, Neuroscience & Biobehavioral Reviews.

[33]  Cameron S Carter,et al.  Development of action monitoring through adolescence into adulthood: ERP and source localization. , 2007, Developmental science.

[34]  Michael X. Cohen,et al.  Error-related medial frontal theta activity predicts cingulate-related structural connectivity , 2011, NeuroImage.

[35]  D. Joel,et al.  The connections of the dopaminergic system with the striatum in rats and primates: an analysis with respect to the functional and compartmental organization of the striatum , 2000, Neuroscience.

[36]  Michael X. Cohen,et al.  Oscillatory Activity and Phase–Amplitude Coupling in the Human Medial Frontal Cortex during Decision Making , 2009, Journal of Cognitive Neuroscience.

[37]  J. Bhattacharya,et al.  High-Learners Present Larger Mid-Frontal Theta Power and Connectivity in Response to Incorrect Performance Feedback , 2013, The Journal of Neuroscience.

[38]  U. Sailer,et al.  Effects of learning on feedback-related brain potentials in a decision-making task , 2010, Brain Research.

[39]  J. Borst,et al.  Electrical activity of the cingulate cortex. I. Generating mechanisms and relations to behavior , 1987, Brain Research.

[40]  Jeffrey L. Birk,et al.  Neural responses to negative feedback are related to negative emotionality in healthy adults. , 2012, Social cognitive and affective neuroscience.

[41]  Timothy E. J. Behrens,et al.  Choice, uncertainty and value in prefrontal and cingulate cortex , 2008, Nature Neuroscience.

[42]  Emanuel Donchin,et al.  Neural response to action and reward prediction errors: Comparing the error-related negativity to behavioral errors and the feedback-related negativity to reward prediction violations. , 2011, Psychophysiology.

[43]  J. Born,et al.  EEG synchronization upon reward in man , 2002, Clinical Neurophysiology.

[44]  Roshan Cools,et al.  Feedback-related Negativity Codes Prediction Error but Not Behavioral Adjustment during Probabilistic Reversal Learning , 2011, Journal of Cognitive Neuroscience.

[45]  Donald G. Burkhard,et al.  Effect of Film Feedback on Learning the Motor Skills of Karate , 1967, Perceptual and motor skills.

[46]  C. Braun,et al.  Event-Related Brain Potentials Following Incorrect Feedback in a Time-Estimation Task: Evidence for a Generic Neural System for Error Detection , 1997, Journal of Cognitive Neuroscience.

[47]  K. R. Ridderinkhof,et al.  Aging affects medial but not anterior frontal learning-related theta oscillations , 2014, Neurobiology of Aging.

[48]  E. Halgren,et al.  Responses of Human Anterior Cingulate Cortex Microdomains to Error Detection, Conflict Monitoring, Stimulus-Response Mapping, Familiarity, and Orienting , 2005, The Journal of Neuroscience.

[49]  Yael Arbel,et al.  The Good, the Bad, or the Useful? The Examination of the Relationship between the Feedback-related Negativity (FRN) and Long-term Learning Outcomes , 2013, Journal of Cognitive Neuroscience.

[50]  T. Schormann,et al.  The Effect of Verbal Feedback on Motor Learning—A PET Study , 2000, NeuroImage.

[51]  A. Engel,et al.  Beta-band oscillations—signalling the status quo? , 2010, Current Opinion in Neurobiology.

[52]  Keiji Tanaka,et al.  Medial prefrontal cell activity signaling prediction errors of action values , 2007, Nature Neuroscience.

[53]  Antoni Rodríguez-Fornells,et al.  Brain oscillatory activity associated with task switching and feedback processing , 2011, Cognitive, Affective, & Behavioral Neuroscience.

[54]  Tobias U. Hauser,et al.  The feedback-related negativity (FRN) revisited: New insights into the localization, meaning and network organization , 2014, NeuroImage.

[55]  Maren Grigutsch,et al.  EEG oscillations and wavelet analysis , 2005 .

[56]  Ian W. Eisenberg,et al.  Frontal Theta Overrides Pavlovian Learning Biases , 2013, The Journal of Neuroscience.

[57]  John J. B. Allen,et al.  Prelude to and Resolution of an Error: EEG Phase Synchrony Reveals Cognitive Control Dynamics during Action Monitoring , 2009, The Journal of Neuroscience.

[58]  J. Hohnsbein,et al.  Effects of crossmodal divided attention on late ERP components. II. Error processing in choice reaction tasks. , 1991, Electroencephalography and clinical neurophysiology.

[59]  David Cucurell,et al.  Human oscillatory activity associated to reward processing in a gambling task , 2008, Neuropsychologia.

[60]  Hakwan Lau,et al.  Prior Expectation Modulates the Interaction between Sensory and Prefrontal Regions in the Human Brain , 2011, The Journal of Neuroscience.

[61]  Timothy E. J. Behrens,et al.  Optimal decision making and the anterior cingulate cortex , 2006, Nature Neuroscience.

[62]  Zhiheng Zhou,et al.  To do or not to do? Action enlarges the FRN and P300 effects in outcome evaluation , 2010, Neuropsychologia.

[63]  David Goodman,et al.  Performance Monitoring in the Anterior Cingulate is Not All Error Related: Expectancy Deviation and the Representation of Action-Outcome Associations , 2007, Journal of Cognitive Neuroscience.

[64]  Matthew S. Tata,et al.  Right frontal cortex generates reward-related theta-band oscillatory activity , 2009, NeuroImage.

[65]  M. Walton,et al.  Interactions between decision making and performance monitoring within prefrontal cortex , 2004, Nature Neuroscience.

[66]  Gregory Ashby,et al.  A neuropsychological theory of multiple systems in category learning. , 1998, Psychological review.

[67]  Clay B. Holroyd,et al.  Errors in reward prediction are re£ected in the event-related brain potential , 2003 .

[68]  R. Baker,et al.  When is an error not a prediction error? An electrophysiological investigation , 2009, Cognitive, affective & behavioral neuroscience.

[69]  Clay B. Holroyd,et al.  It's worse than you thought: the feedback negativity and violations of reward prediction in gambling tasks. , 2007, Psychophysiology.

[70]  W. Schultz Getting Formal with Dopamine and Reward , 2002, Neuron.

[71]  Timothy Edward John Behrens,et al.  Contrasting roles for cingulate and orbitofrontal cortex in decisions and social behaviour , 2007, Trends in Cognitive Sciences.

[72]  Josep Marco-Pallarés,et al.  Genetic Variability in the Dopamine System (Dopamine Receptor D4, Catechol-O-Methyltransferase) Modulates Neurophysiological Responses to Gains and Losses , 2009, Biological Psychiatry.

[73]  Michael X. Cohen,et al.  Dynamic Interactions between Large-Scale Brain Networks Predict Behavioral Adaptation after Perceptual Errors , 2012, Cerebral cortex.

[74]  I. Daum,et al.  Processing of expected and unexpected monetary performance outcomes in healthy older subjects. , 2011, Behavioral neuroscience.

[75]  Will F W Wu,et al.  Contextual interference and augmented feedback: is there an additive effect for motor learning? , 2011, Human movement science.

[76]  Michael J. Frank,et al.  Error-Related Negativity Predicts Reinforcement Learning and Conflict Biases , 2005, Neuron.

[77]  M. Botvinick,et al.  Anterior cingulate cortex, error detection, and the online monitoring of performance. , 1998, Science.

[78]  J. Berke,et al.  Fast oscillations in cortical‐striatal networks switch frequency following rewarding events and stimulant drugs , 2009, The European journal of neuroscience.

[79]  Michael X. Cohen,et al.  Reward expectation modulates feedback-related negativity and EEG spectra , 2007, NeuroImage.

[80]  Michael X. Cohen,et al.  Frontal Oscillatory Dynamics Predict Feedback Learning and Action Adjustment , 2011, Journal of Cognitive Neuroscience.

[81]  A. Luft,et al.  Predictive value and reward in implicit classification learning , 2013, Human brain mapping.

[82]  Yaniv Assaf,et al.  Learning in the Fast Lane: New Insights into Neuroplasticity , 2012, Neuron.

[83]  David Papo,et al.  Modulation of late alpha band oscillations by feedback in a hypothesis testing paradigm. , 2007, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[84]  C. Shea,et al.  Frequent feedback enhances complex motor skill learning. , 1998, Journal of motor behavior.

[85]  C. Carter,et al.  The Timing of Action-Monitoring Processes in the Anterior Cingulate Cortex , 2002, Journal of Cognitive Neuroscience.

[86]  Michael X. Cohen,et al.  Neurocomputational mechanisms of reinforcement-guided learning in humans: A review , 2008, Cognitive, affective & behavioral neuroscience.

[87]  Thomas F Münte,et al.  Internal and external information in error processing , 2008, BMC Neuroscience.

[88]  Michela Balconi,et al.  Error monitoring functions in response to an external feedback when an explicit judgement is required: ERP modulation and cortical source localisation. , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.