Prefrontal Cortex Activation Upon a Demanding Virtual Hand-Controlled Task: A New Frontier for Neuroergonomics

Functional near-infrared spectroscopy (fNIRS) is a non-invasive vascular-based functional neuroimaging technology that can assess, simultaneously from multiple cortical areas, concentration changes in oxygenated-deoxygenated hemoglobin at the level of the cortical microcirculation blood vessels. fNIRS, with its high degree of ecological validity and its very limited requirement of physical constraints to subjects, could represent a valid tool for monitoring cortical responses in the research field of neuroergonomics. In virtual reality (VR) real situations can be replicated with greater control than those obtainable in the real world. Therefore, VR is the ideal setting where studies about neuroergonomics applications can be performed. The aim of the present study was to investigate, by a 20-channel fNIRS system, the dorsolateral/ventrolateral prefrontal cortex (DLPFC/VLPFC) in subjects while performing a demanding VR hand-controlled task (HCT). Considering the complexity of the HCT, its execution should require the attentional resources allocation and the integration of different executive functions. The HCT simulates the interaction with a real, remotely-driven, system operating in a critical environment. The hand movements were captured by a high spatial and temporal resolution 3-dimensional (3D) hand-sensing device, the LEAP motion controller, a gesture-based control interface that could be used in VR for tele-operated applications. Fifteen University students were asked to guide, with their right hand/forearm, a virtual ball (VB) over a virtual route (VROU) reproducing a 42 m narrow road including some critical points. The subjects tried to travel as long as possible without making VB fall. The distance traveled by the guided VB was 70.2 ± 37.2 m. The less skilled subjects failed several times in guiding the VB over the VROU. Nevertheless, a bilateral VLPFC activation, in response to the HCT execution, was observed in all the subjects. No correlation was found between the distance traveled by the guided VB and the corresponding cortical activation. These results confirm the suitability of fNIRS technology to objectively evaluate cortical hemodynamic changes occurring in VR environments. Future studies could give a contribution to a better understanding of the cognitive mechanisms underlying human performance either in expert or non-expert operators during the simulation of different demanding/fatiguing activities.

[1]  Barry H. Kantowitz,et al.  Mental Workload , 2020, Encyclopedia of Behavioral Medicine.

[2]  Y. Takeuchi Change in Blood Volume in the Brain during a Simulated Aircraft Landing Task , 2000 .

[3]  A. Baddeley Working memory: looking back and looking forward , 2003, Nature Reviews Neuroscience.

[4]  H. Tsunashima,et al.  Measurement of train driver's brain activity by functional near-infrared spectroscopy (fNIRS) , 2004, The Second Asian and Pacific Rim Symposium on Biophotonics, 2004. APBP 2004..

[5]  M. P. Matud,et al.  Gender differences in stress and coping styles , 2004 .

[6]  Yuji Saruta,et al.  The state of , 2005 .

[7]  Raja Parasuraman,et al.  Neuroergonomics: The Brain at Work , 2006 .

[8]  S. Bunce,et al.  Functional near-infrared spectroscopy , 2006, IEEE Engineering in Medicine and Biology Magazine.

[9]  Joseph K. Kearney,et al.  Virtual Reality and Neuroergonomics , 2009, Neuroergonomics.

[10]  U. Halsband,et al.  Motor learning in man: A review of functional and clinical studies , 2006, Journal of Physiology-Paris.

[11]  David Badre,et al.  Left ventrolateral prefrontal cortex and the cognitive control of memory , 2007, Neuropsychologia.

[12]  Ichiro Miyai,et al.  Frontal regions involved in learning of motor skill—A functional NIRS study , 2007, NeuroImage.

[13]  J. Tanji,et al.  Role of the lateral prefrontal cortex in executive behavioral control. , 2008, Physiological reviews.

[14]  Kaoru Takakusaki,et al.  Forebrain control of locomotor behaviors , 2008, Brain Research Reviews.

[15]  Scott Sinnett,et al.  Role of the lateral prefrontal cortex in visual object-based selective attention , 2008, Experimental Brain Research.

[16]  M. Corbetta,et al.  The Reorienting System of the Human Brain: From Environment to Theory of Mind , 2008, Neuron.

[17]  Masaru Mimura,et al.  Detection of hypofrontality in drivers with Alzheimer's disease by near-infrared spectroscopy , 2009, Neuroscience Letters.

[18]  Sarah E. Donohue,et al.  Controlled retrieval and selection of action-relevant knowledge mediated by partially overlapping regions in left ventrolateral prefrontal cortex , 2009, NeuroImage.

[19]  Hitoshi Tsunashima,et al.  Measurement of Brain Function of Car Driver Using Functional Near-Infrared Spectroscopy (fNIRS) , 2009, Comput. Intell. Neurosci..

[20]  K. E. Steele,et al.  REVIEW PAPER , 2010, Veterinary pathology.

[21]  G. Dumont,et al.  Wavelet based motion artifact removal for Functional Near Infrared Spectroscopy , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[22]  A. Gunji,et al.  Scanning Strategies Do Not Modulate Face Identification: Eye-Tracking and Near-Infrared Spectroscopy Study , 2010, PloS one.

[23]  David A. Boas,et al.  Improved recovery of the hemodynamic response in diffuse optical imaging using short optode separations and state-space modeling , 2011, NeuroImage.

[24]  Guang-Zhong Yang,et al.  The ergonomics of natural orifice translumenal endoscopic surgery (NOTES) navigation in terms of performance, stress, and cognitive behavior. , 2011, Surgery.

[25]  Kelvin S. Oie,et al.  Cognition in action: imaging brain/body dynamics in mobile humans , 2011, Reviews in the neurosciences.

[26]  Luciano Gamberini,et al.  An exploratory fNIRS study with immersive virtual reality: a new method for technical implementation , 2011, Front. Hum. Neurosci..

[27]  Jeannette R. Mahoney,et al.  fNIRS study of walking and walking while talking in young and old individuals. , 2011, The journals of gerontology. Series A, Biological sciences and medical sciences.

[28]  Raja Parasuraman,et al.  Neuroergonomics , 2011 .

[29]  Guang-Zhong Yang,et al.  Assessment of the cerebral cortex during motor task behaviours in adults: A systematic review of functional near infrared spectroscopy (fNIRS) studies , 2011, NeuroImage.

[30]  Manfred Morari,et al.  The effects of manipulation of visual feedback in virtual reality on cortical activity: A pilot study , 2011, 2011 International Conference on Virtual Rehabilitation.

[31]  Hasan Ayaz,et al.  Using MazeSuite and Functional Near Infrared Spectroscopy to Study Learning in Spatial Navigation , 2011, Journal of visualized experiments : JoVE.

[32]  Hasan Ayaz,et al.  Optical brain monitoring for operator training and mental workload assessment , 2012, NeuroImage.

[33]  Jun Tanji,et al.  Distinct Information Representation and Processing for Goal-Directed Behavior in the Dorsolateral and Ventrolateral Prefrontal Cortex and the Dorsal Premotor Cortex , 2012, The Journal of Neuroscience.

[34]  Theodore Huppert,et al.  Functional near-infrared spectroscopy (fNIRS) of brain function during active balancing using a video game system. , 2012, Gait & posture.

[35]  Christa Neuper,et al.  The interplay of prefrontal and sensorimotor cortices during inhibitory control of learned motor behavior , 2012, Front. Neuroeng..

[36]  Kynan Eng,et al.  Trial-to-trial variability differentiates motor imagery during observation between low versus high responders: A functional near-infrared spectroscopy study , 2012, Behavioural Brain Research.

[37]  Keiichi Watanuki,et al.  Measurement of Brain Activity under Virtual Reality Skills Training Using Near-Infrared Spectroscopy , 2012 .

[38]  K. Izzetoglu,et al.  Monitoring expertise development during simulated UAV piloting tasks using optical brain imaging , 2012, 2012 IEEE Aerospace Conference.

[39]  Theodore Huppert,et al.  Measurement of brain activation during an upright stepping reaction task using functional near‐infrared spectroscopy , 2013, Human brain mapping.

[40]  Marco Ferrari,et al.  Prefrontal Cortex Activated Bilaterally by a Tilt Board Balance Task: A Functional Near-Infrared Spectroscopy Study in a Semi-Immersive Virtual Reality Environment , 2013, Brain Topography.

[41]  Ranjana K. Mehta,et al.  Neuroergonomics: a review of applications to physical and cognitive work , 2013, Front. Hum. Neurosci..

[42]  Matthias Scheutz,et al.  Functional near-infrared spectroscopy in human-robot interaction , 2013, HRI 2013.

[43]  Neil A. Dodgson,et al.  Optical devices: 3D without the glasses , 2013, Nature.

[44]  D. Kiper,et al.  Testing the potential of a virtual reality neurorehabilitation system during performance of observation, imagery and imitation of motor actions recorded by wireless functional near-infrared spectroscopy (fNIRS) , 2010, Journal of NeuroEngineering and Rehabilitation.

[45]  Marco Ferrari,et al.  Prefrontal cortex activation during story encoding/retrieval: a multi-channel functional near-infrared spectroscopy study , 2013, Front. Hum. Neurosci..

[46]  R. Gentili,et al.  Functional near-infrared spectroscopy-based correlates of prefrontal cortical dynamics during a cognitive-motor executive adaptation task , 2013, Front. Hum. Neurosci..

[47]  Gérard Dray,et al.  NIRS-measured prefrontal cortex activity in neuroergonomics: strengths and weaknesses , 2013, Front. Hum. Neurosci..

[48]  E. Hoshi Cortico-basal ganglia networks subserving goal-directed behavior mediated by conditional visuo-goal association , 2013, Front. Neural Circuits.

[49]  Gérard Dray,et al.  Prefrontal cortex activity during motor tasks with additional mental load requiring attentional demand: A near-infrared spectroscopy study , 2013, Neuroscience Research.

[50]  R. Parasuraman,et al.  Continuous monitoring of brain dynamics with functional near infrared spectroscopy as a tool for neuroergonomic research: empirical examples and a technological development , 2013, Front. Hum. Neurosci..

[51]  Pyung-Hun Chang,et al.  The cortical activation pattern by a rehabilitation robotic hand: a functional NIRS study , 2014, Front. Hum. Neurosci..

[52]  Diane L Damiano,et al.  Functional near infrared spectroscopy of the sensory and motor brain regions with simultaneous kinematic and EMG monitoring during motor tasks. , 2014, Journal of visualized experiments : JoVE.

[53]  G. Derosière,et al.  Similar scaling of contralateral and ipsilateral cortical responses during graded unimanual force generation , 2014, NeuroImage.

[54]  Nir Giladi,et al.  Increased frontal brain activation during walking while dual tasking: an fNIRS study in healthy young adults , 2014, Journal of NeuroEngineering and Rehabilitation.

[55]  Marco Ferrari,et al.  A semi-immersive virtual reality incremental swing balance task activates prefrontal cortex: A functional near-infrared spectroscopy study , 2014, NeuroImage.

[56]  Daniel P Ferris,et al.  Imaging natural cognition in action. , 2014, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[57]  Ulf Ahlstrom,et al.  Cognitive Workload and Learning Assessment During the Implementation of a Next-Generation Air Traffic Control Technology Using Functional Near-Infrared Spectroscopy , 2014, IEEE Transactions on Human-Machine Systems.

[58]  Ning Zhang,et al.  vDesign: a CAVE-based virtual design environment using hand interactions , 2014, Journal on Multimodal User Interfaces.

[59]  T. Ono,et al.  Selective Medial Prefrontal Cortex Responses During Live Mutual Gaze Interactions in Human Infants: An fNIRS Study , 2015, Brain Topography.

[60]  Martin Wolf,et al.  A review on continuous wave functional near-infrared spectroscopy and imaging instrumentation and methodology , 2014, NeuroImage.

[61]  Akinori Nagano,et al.  An fNIRS-Based Study on Prefrontal Cortex Activity during a Virtual Shopping Test with Different Task Difficulties in Brain-Damaged Patients , 2014 .

[62]  Matthias Scheutz,et al.  What we can and cannot (yet) do with functional near infrared spectroscopy , 2014, Front. Neurosci..

[63]  A. Enis Çetin,et al.  Hand gesture based remote control system using infrared sensors and a camera , 2014, IEEE Transactions on Consumer Electronics.

[64]  Gary E. Strangman,et al.  Twenty-four-hour ambulatory recording of cerebral hemodynamics, systemic hemodynamics, electrocardiography, and actigraphy during people’s daily activities , 2014, Journal of biomedical optics.

[65]  Jacques Duysens,et al.  Cortical control of normal gait and precision stepping: An fNIRS study , 2014, NeuroImage.

[66]  Daniel Afergan,et al.  Using fNIRS to Measure Mental Workload in the Real World , 2014 .

[67]  Parth Rajesh Desai,et al.  A Review Paper on Oculus Rift-A Virtual Reality Headset , 2014, ArXiv.

[68]  S. Tremblay,et al.  Using near infrared spectroscopy and heart rate variability to detect mental overload , 2014, Behavioural Brain Research.

[69]  Daniel Afergan,et al.  Phylter: A System for Modulating Notifications in Wearables Using Physiological Sensing , 2015, HCI.

[70]  Raja Parasuraman,et al.  Wearable functional near infrared spectroscopy (fNIRS) and transcranial direct current stimulation (tDCS): expanding vistas for neurocognitive augmentation , 2015, Front. Syst. Neurosci..

[71]  Gerhard Rinkenauer,et al.  Evaluation of the Leap Motion Controller as a New Contact-Free Pointing Device , 2014, Sensors.

[72]  Dominic Heger,et al.  Toward a Wireless Open Source Instrument: Functional Near-infrared Spectroscopy in Mobile Neuroergonomics and BCI Applications , 2015, Front. Hum. Neurosci..

[73]  Keum-Shik Hong,et al.  fNIRS-based brain-computer interfaces: a review , 2015, Front. Hum. Neurosci..

[74]  Peter A Hancock,et al.  State of science: mental workload in ergonomics , 2015, Ergonomics.

[75]  Stephen R Mitroff,et al.  Mapping the structure of perceptual and visual-motor abilities in healthy young adults. , 2015, Acta psychologica.

[76]  Thibault Gateau,et al.  Real-Time State Estimation in a Flight Simulator Using fNIRS , 2015, PloS one.

[77]  Mengyin Fu,et al.  Hand Gesture Based Robot Control System Using Leap Motion , 2015, ICIRA.

[78]  Yukang Liu,et al.  Toward Welding Robot With Human Knowledge: A Remotely-Controlled Approach , 2015, IEEE Transactions on Automation Science and Engineering.

[79]  Yumie Ono,et al.  fMRI Validation of fNIRS Measurements During a Naturalistic Task , 2015, Journal of visualized experiments : JoVE.

[80]  Ara W. Darzi,et al.  The impact of expert visual guidance on trainee visual search strategy, visual attention and motor skills , 2015, Front. Hum. Neurosci..

[81]  Joyce Fung,et al.  Cortical mechanisms underlying sensorimotor enhancement promoted by walking with haptic inputs in a virtual environment. , 2015, Progress in brain research.

[82]  J. Hirsch,et al.  Motor learning and modulation of prefrontal cortex: an fNIRS assessment , 2015, Journal of neural engineering.

[83]  Lee Jun Wei,et al.  LEAP MOTION UNDERWATER ROBOTIC ARM CONTROL , 2015 .

[84]  Mary L. Cummings,et al.  Investigating Mental Workload Changes in a Long Duration Supervisory Control Task , 2015, Interact. Comput..

[85]  Masakatsu G. Fujie,et al.  Brain activation in parietal area during manipulation with a surgical robot simulator , 2015, International Journal of Computer Assisted Radiology and Surgery.

[86]  Tao Liu,et al.  Near-infrared spectroscopy as a tool for driving research , 2016, Ergonomics.