Psychophysiological Markers of Performance and Learning during Simulated Marksmanship in Immersive Virtual Reality

Abstract The fusion of immersive virtual reality, kinematic movement tracking, and EEG offers a powerful test bed for naturalistic neuroscience research. Here, we combined these elements to investigate the neuro-behavioral mechanisms underlying precision visual–motor control as 20 participants completed a three-visit, visual–motor, coincidence-anticipation task, modeled after Olympic Trap Shooting and performed in immersive and interactive virtual reality. Analyses of the kinematic metrics demonstrated learning of more efficient movements with significantly faster hand RTs, earlier trigger response times, and higher spatial precision, leading to an average of 13% improvement in shot scores across the visits. As revealed through spectral and time-locked analyses of the EEG beta band (13–30 Hz), power measured prior to target launch and visual-evoked potential amplitudes measured immediately after the target launch correlated with subsequent reactive kinematic performance in the shooting task. Moreover, both launch-locked and shot/feedback-locked visual-evoked potentials became earlier and more negative with practice, pointing to neural mechanisms that may contribute to the development of visual–motor proficiency. Collectively, these findings illustrate EEG and kinematic biomarkers of precision motor control and changes in the neurophysiological substrates that may underlie motor learning.

[1]  Christa Neuper,et al.  Distinct β Band Oscillatory Networks Subserving Motor and Cognitive Control during Gait Adaptation , 2016, The Journal of Neuroscience.

[2]  A. Mierau,et al.  The Speed of Neural Visual Motion Perception and Processing Determines the Visuomotor Reaction Time of Young Elite Table Tennis Athletes , 2019, Front. Behav. Neurosci..

[3]  Scott T. Grafton,et al.  Forward modeling allows feedback control for fast reaching movements , 2000, Trends in Cognitive Sciences.

[4]  M. Kuba,et al.  Motion-onset VEPs: Characteristics, methods, and diagnostic use , 2007, Vision Research.

[5]  W. Klimesch EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis , 1999, Brain Research Reviews.

[6]  T. Komiyama,et al.  Circuit Mechanisms of Sensorimotor Learning , 2016, Neuron.

[7]  Scott Makeig Mind Monitoring via Mobile Brain-Body Imaging , 2009, HCI.

[8]  Manan Shah,et al.  Application on Virtual Reality for Enhanced Education Learning, Military Training and Sports , 2019, Augmented Human Research.

[9]  I M Franks,et al.  Preprogramming vs. on-line control in simple movement sequences. , 1991, Acta psychologica.

[10]  John R. Anderson,et al.  Learning from experience: Event-related potential correlates of reward processing, neural adaptation, and behavioral choice , 2012, Neuroscience & Biobehavioral Reviews.

[11]  Doug A. Bowman,et al.  Feasibility of Training Athletes for High-Pressure Situations Using Virtual Reality , 2014, IEEE Transactions on Visualization and Computer Graphics.

[12]  Kelvin S. Oie,et al.  Cognition in action: imaging brain/body dynamics in mobile humans , 2011, Reviews in the neurosciences.

[13]  R. Dodhia A Review of Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences (3rd ed.) , 2005 .

[14]  Kait Clark,et al.  Improvement in Visual Search with Practice: Mapping Learning-Related Changes in Neurocognitive Stages of Processing , 2015, The Journal of Neuroscience.

[15]  Christopher M. Janelle,et al.  Visual Attention and Brain Processes That Underlie Expert Performance: Implications for Sport and Military Psychology , 2008 .

[16]  P. Fitts The information capacity of the human motor system in controlling the amplitude of movement. , 1954, Journal of experimental psychology.

[17]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[18]  Tsung-Min Hung,et al.  Electroencephalographic Studies of Skilled Psychomotor Performance , 2004, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[19]  Ferran Argelaguet,et al.  A Methodology for Introducing Competitive Anxiety and Pressure in VR Sports Training , 2015, Front. Robot. AI.

[20]  Nicolas Robitaille,et al.  Distinguishing between lateralized and nonlateralized brain activity associated with visual short-term memory: fMRI, MEG, and EEG evidence from the same observers , 2010, NeuroImage.

[21]  Heather Colquhoun,et al.  Promoting Therapists’ Use of Motor Learning Strategies within Virtual Reality-Based Stroke Rehabilitation , 2016, PloS one.

[22]  Tatiana de Paula Oliveira,et al.  Motor learning, retention and transfer after virtual-reality-based training in Parkinson's disease--effect of motor and cognitive demands of games: a longitudinal, controlled clinical study. , 2012, Physiotherapy.

[23]  Alex R. Wade,et al.  Cue-Invariant Networks for Figure and Background Processing in Human Visual Cortex , 2006, The Journal of Neuroscience.

[24]  Maarten A. S. Boksem,et al.  The Importance of Failure: Feedback Related Negativity Predicts Motor Learning Efficiency , 2009, NeuroImage.

[25]  D. Elliott,et al.  125 years of perceptual-motor skill research. , 2012, The American journal of psychology.

[26]  Rajesh Aggarwal,et al.  An Evidence-Based Virtual Reality Training Program for Novice Laparoscopic Surgeons , 2006, Annals of surgery.

[27]  M. Arns,et al.  Golf performance enhancement and real-life neurofeedback training using personalized event-locked EEG profiles , 2008 .

[28]  Hrishikesh M. Rao,et al.  Sensorimotor Learning during a Marksmanship Task in Immersive Virtual Reality , 2018, Front. Psychol..

[29]  A Daffertshofer,et al.  Differential modulations of ipsilateral and contralateral beta (de)synchronization during unimanual force production , 2012, The European journal of neuroscience.

[30]  T. Sejnowski,et al.  Linking brain, mind and behavior. , 2008, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[31]  Yongmin Chang,et al.  Stronger activation and deactivation in archery experts for differential cognitive strategy in visuospatial working memory processing , 2012, Behavioural Brain Research.

[32]  David E. Meyer,et al.  Speed—Accuracy Tradeoffs in Aimed Movements: Toward a Theory of Rapid Voluntary Action , 2018, Attention and Performance XIII.

[33]  Kait Clark,et al.  Visual search performance is predicted by both prestimulus and poststimulus electrical brain activity , 2016, Scientific Reports.

[34]  C. Hillman,et al.  An electrocortical comparison of executed and rejected shots in skilled marksmen , 2000, Biological Psychology.

[35]  Anthony M Norcia,et al.  Attentive and pre-attentive aspects of figural processing. , 2009, Journal of vision.

[36]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[37]  J. Kaiser,et al.  Human gamma-frequency oscillations associated with attention and memory , 2007, Trends in Neurosciences.

[38]  Justin M. Ales,et al.  The Time Course of Segmentation and Cue-Selectivity in the Human Visual Cortex , 2012, PloS one.

[39]  James L. Lyons,et al.  Goal-directed aiming: two components but multiple processes. , 2010, Psychological bulletin.

[40]  Per B. Brockhoff,et al.  lmerTest Package: Tests in Linear Mixed Effects Models , 2017 .

[41]  Leslie M. Collins,et al.  Neurophysiology of Visual-Motor Learning During a Simulated Marksmanship Task in Immersive Virtual Reality , 2018, 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[42]  J. Krakauer,et al.  Inside the brain of an elite athlete: the neural processes that support high achievement in sports , 2009, Nature Reviews Neuroscience.

[43]  J. Gruzelier EEG-neurofeedback for optimising performance. I: A review of cognitive and affective outcome in healthy participants , 2014, Neuroscience & Biobehavioral Reviews.

[44]  Benno M Nigg,et al.  Changes in cortical activity measured with EEG during a high-intensity cycling exercise. , 2016, Journal of neurophysiology.

[45]  Franck Vidal,et al.  Sequential adjustments before and after partial errors , 2009, Psychonomic bulletin & review.

[46]  Klaus Gramann,et al.  Mobile Brain/Body Imaging (MoBI) of Physical Interaction with Dynamically Moving Objects , 2016, Front. Hum. Neurosci..

[47]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[48]  Karl F. Van Orden,et al.  A neuroscience approach to optimizing brain resources for human performance in extreme environments , 2009, Neuroscience & Biobehavioral Reviews.

[49]  Maren S Fragala,et al.  Biomarkers in Sports and Exercise: Tracking Health, Performance, and Recovery in Athletes , 2017, Journal of strength and conditioning research.

[50]  M. Levin,et al.  Emergence of Virtual Reality as a Tool for Upper Limb Rehabilitation: Incorporation of Motor Control and Motor Learning Principles , 2014, Physical Therapy.

[51]  Chris Berka,et al.  Accelerating Training Using Interactive Neuro-Educational Technologies , 2010 .

[52]  Clay B. Holroyd,et al.  The feedback-related negativity reflects the binary evaluation of good versus bad outcomes , 2006, Biological Psychology.

[53]  Thomas Brochier,et al.  Modulations of EEG Beta Power during Planning and Execution of Grasping Movements , 2013, PloS one.

[54]  M. Iacoboni,et al.  Golf putt outcomes are predicted by sensorimotor cerebral EEG rhythms , 2008, The Journal of physiology.

[55]  Eric D. Ragan,et al.  The Effects of Higher Levels of Immersion on Procedure Memorization Performance and Implications for Educational Virtual Environments , 2010, PRESENCE: Teleoperators and Virtual Environments.

[56]  Michael I. Jordan,et al.  An internal model for sensorimotor integration. , 1995, Science.

[57]  Nigel W. John,et al.  A review of virtual environments for training in ball sports , 2012, Comput. Graph..

[58]  S. Swinnen,et al.  Understanding bimanual coordination across small time scales from an electrophysiological perspective , 2014, Neuroscience & Biobehavioral Reviews.

[59]  Thomas F. Quatieri,et al.  Predicting Cognitive Load and Operational Performance in a Simulated Marksmanship Task , 2020, Frontiers in Human Neuroscience.

[60]  James S. P. Macdonald,et al.  Trial-by-Trial Variations in Subjective Attentional State are Reflected in Ongoing Prestimulus EEG Alpha Oscillations , 2011, Front. Psychology.

[61]  Carolina Cruz-Neira,et al.  Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE , 2023 .

[62]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[63]  Guy Cheron,et al.  Brain Oscillations in Sport: Toward EEG Biomarkers of Performance , 2016, Front. Psychol..

[64]  J. Lundbye-Jensen,et al.  Changes in corticospinal drive to spinal motoneurones following visuo‐motor skill learning in humans , 2006, The Journal of physiology.

[65]  Fang Wang,et al.  The N2pc Is Increased by Perceptual Learning but Is Unnecessary for the Transfer of Learning , 2012, PloS one.

[66]  L. Appelbaum,et al.  Sports vision training: A review of the state-of-the-art in digital training techniques , 2016 .

[67]  A. Mierau,et al.  Visual Motion Processing Subserves Faster Visuomotor Reaction in Badminton Players , 2017, Medicine and science in sports and exercise.

[68]  Rob Gray,et al.  Transfer of Training from Virtual to Real Baseball Batting , 2017, Front. Psychol..

[69]  Michael X. Cohen,et al.  Reward expectation modulates feedback-related negativity and EEG spectra , 2007, NeuroImage.

[70]  Steven J. Petruzzello,et al.  Effects of learning on electroencephalographic and electrocardiographic patterns in novice archers. , 1994 .

[71]  Jonathan R. Folstein,et al.  Anticipation in Sharp Shooting: Cognitive Structures in Detecting Performance Errors , 2019, Psychology of Sport and Exercise.

[72]  David F Stodden,et al.  Impulse-Variability Theory: Implications for Ballistic, Multijoint Motor Skill Performance , 2011, Journal of motor behavior.

[73]  D. Wolpert,et al.  Computations underlying sensorimotor learning , 2016, Current Opinion in Neurobiology.

[74]  P. Rabbitt Errors and error correction in choice-response tasks. , 1966, Journal of experimental psychology.

[75]  William H Warren,et al.  Catching fly balls in virtual reality: a critical test of the outfielder problem. , 2009, Journal of vision.

[76]  Sean D. Kristjansson,et al.  Rapid discrimination of visual scene content in the human brain , 2006, Brain Research.