Mental workload characteristics of manipulator teleoperators with different spatial cognitive abilities

The main research on manipulator teleoperation includes robust of high-degree of freedom manipulators, sensor measurement accuracy, time delay, and mechanical structure design. Increased mental capacity requirements for complex assignments result in an increased mental workload. Spatial cognitive ability was considered to be the key factor affecting teleoperation performance. To accomplish this, we had 50 participants performed teleoperation while recorded their electroencephalogram. Electroencephalogram data of each task were divided into two periods, which correspond to the observation and large-scale transfer stages of teleoperation, respectively (period 1) and adjust the attitude of the manipulator to approach and align with the target stage (period 2). Brain topographic maps of period 1 (period 1 wavelet packet energy minus resting state wavelet packet energy) and period 2 (period 2 wavelet packet energy minus resting state wavelet packet energy) show that the frontal, central, and occipital regions are the main working areas of low spatial cognitive operators in period 1, while the frontal, central, and occipital regions are the main working areas of high spatial cognitive operators in period 1. The main changes in period 2 were frontal, central, parietal, and occipital regions. This study has implication for the analysis of electroencephalogram signal characteristics of mental workload in different populations to improve operators’ well-being and safety at teleoperation work.

[1]  J. Edward Colgate,et al.  Improving teleoperation: reducing mental rotations and translations , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[2]  Ilona Papousek,et al.  The time-course of EEG alpha power changes in creative ideation , 2014, Front. Hum. Neurosci..

[3]  C. Mélan,et al.  What is the relationship between mental workload factors and cognitive load types? , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[4]  Gerald Matthews,et al.  Mental workload and stress perceived by novice operators in the laparoscopic and robotic minimally invasive surgical interfaces. , 2012, Journal of endourology.

[5]  Christopher D. Wickens,et al.  The Design and Evaluation of Visual and Tactile Warnings in Support of Space Teleoperation , 2012 .

[6]  H. Heuer,et al.  Frontal theta activity reflects distinct aspects of mental fatigue , 2014, Biological Psychology.

[7]  N. Henderson,et al.  PREDICTING LONG‐TERM FIREFIGHTER PERFORMANCE FROM COGNITIVE AND PHYSICAL ABILITY MEASURES , 2010 .

[8]  Soheil Ganjefar,et al.  Position and force tracking in nonlinear teleoperation systems with sandwich linearity in actuators and time-varying delay , 2017 .

[9]  Su Yang,et al.  Wavelet-Based EEG Preprocessing for Biometric Applications , 2013, 2013 Fourth International Conference on Emerging Security Technologies.

[10]  C. Richards,et al.  Brain activity during visual versus kinesthetic imagery: An fMRI study , 2009, Human brain mapping.

[11]  Yijing Zhang,et al.  Association of Individual Characteristics with Teleoperation Performance. , 2016, Aerospace medicine and human performance.

[12]  Hironao Yamada,et al.  Development of a Hydraulic Tele-Operated Construction Robot using Virtual Reality , 2003 .

[13]  Michael A. Rupp,et al.  Evaluating input device usability as a function of task difficulty in a tracking task , 2015, Ergonomics.

[14]  James C. Christensen,et al.  The effects of day-to-day variability of physiological data on operator functional state classification , 2012, NeuroImage.

[15]  Kristian Lukander,et al.  Estimating Brain Load from the EEG , 2009, TheScientificWorldJournal.

[16]  Michelle N. Lumicao,et al.  EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. , 2007, Aviation, space, and environmental medicine.

[17]  Complexity Analysis of EEG Under Magnetic Stimulation at Acupoints , 2010, IEEE Transactions on Applied Superconductivity.

[18]  Sara Lal,et al.  Comparing combinations of EEG activity in train drivers during monotonous driving , 2011, Expert Syst. Appl..

[19]  Ahmed A. D. Sarhan,et al.  Improving teleoperation system performance in the presence of estimated external force , 2017 .

[20]  S Bonnet,et al.  Efficient mental workload estimation using task-independent EEG features , 2016, Journal of neural engineering.

[21]  Shuxiang Guo,et al.  Implementation of self-rehabilitation for upper limb based on a haptic device and an exoskeleton device , 2011, 2011 IEEE International Conference on Mechatronics and Automation.

[22]  G. Borghini,et al.  Neuroscience and Biobehavioral Reviews , 2022 .

[23]  Gabriel Curio,et al.  Unsupervised classification of operator workload from brain signals. , 2016, Journal of neural engineering.

[24]  D. R. Montello,et al.  Learning a Spatial Skill for Surgery : How the Contributions of Abilities Change With Practice , 2006 .

[25]  Ying Le-an,et al.  Brief review on physiological and biochemical evaluations of human mental workload , 2012 .

[26]  Terrence Fong,et al.  Advanced Interfaces for Vehicle Teleoperation: Collaborative Control, Sensor Fusion Displays, and Remote Driving Tools , 2001, Auton. Robots.

[27]  R Eyal,et al.  Spatial ability and learning the use of an angled laparoscope in a virtual environment. , 2001, Studies in health technology and informatics.

[28]  Qiang Ji,et al.  Cross-subject workload classification with a hierarchical Bayes model , 2012, NeuroImage.

[29]  G. Glover,et al.  Dissociating Prefrontal and Parietal Cortex Activation during Arithmetic Processing , 2000, NeuroImage.

[30]  M. Hegarty,et al.  A dissociation between mental rotation and perspective-taking spatial abilities , 2004 .

[31]  Ioannis Kompatsiaris,et al.  Brain source localization of MMN and P300 ERPs in mild cognitive impairment and Alzheimer's disease: a high-density EEG approach , 2017, Neurobiology of Aging.

[32]  Mao-Jiun J. Wang,et al.  Effective Indices for Monitoring Mental Workload While Performing Multiple Tasks , 2015, Perceptual and motor skills.

[33]  Leo Gugerty,et al.  Effects of Electronic Map Displays and Individual Differences in Ability on Navigation Performance , 2012, Hum. Factors.

[34]  Aiguo Song,et al.  System centroid position based tipover stability enhancement method for a tracked search and rescue robot , 2014, Adv. Robotics.

[35]  Robert Oostenveld,et al.  Estimating workload using EEG spectral power and ERPs in the n-back task , 2012, Journal of neural engineering.

[36]  Sven Hoffmann,et al.  Towards the measurement of event-related EEG activity in real-life working environments. , 2014, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[37]  P.T.L.M. Van Woerkom,et al.  Invited survey paper on robotic manipulators in space: A dynamics and control perspective , 1995 .

[38]  Alan Natapoff,et al.  Influence of perspective-taking and mental rotation abilities in space teleoperation , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[39]  Tanja Schultz,et al.  Mental workload during n-back task—quantified in the prefrontal cortex using fNIRS , 2014, Front. Hum. Neurosci..

[40]  Gerd Hirzinger,et al.  Toward understanding the effects of visual- and force-feedback on robotic hand grasping performance for space teleoperation , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[41]  Shigeru Haga,et al.  Effects of task difficulty and time‐on‐task on mental workload , 2002 .

[42]  Peter A Hancock,et al.  State of science: mental workload in ergonomics , 2015, Ergonomics.

[43]  M. Hegarty,et al.  Spatial ability, experience, and skill in laparoscopic surgery. , 2004, American journal of surgery.

[44]  Zakiya Alexandra Tomlinson,et al.  Influence of spatial abilities on primary and secondary space telerobotics operator performance , 2009 .

[45]  Sonia Chernova,et al.  A Comparison of Remote Robot Teleoperation Interfaces for General Object Manipulation , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[46]  J. Schoffelen,et al.  Prestimulus Oscillatory Activity in the Alpha Band Predicts Visual Discrimination Ability , 2008, The Journal of Neuroscience.

[47]  John J. B. Allen,et al.  Frontal EEG asymmetry and the behavioral activation and inhibition systems. , 2003, Psychophysiology.

[48]  Fabio Babiloni,et al.  Adaptive Automation Triggered by EEG-Based Mental Workload Index: A Passive Brain-Computer Interface Application in Realistic Air Traffic Control Environment , 2016, Front. Hum. Neurosci..

[49]  Yu Tian,et al.  Predicting performance in manually controlled rendezvous and docking through spatial abilities , 2014 .

[50]  F. Herrmann,et al.  Assessment of mental workload: A new electrophysiological method based on intra-block averaging of ERP amplitudes , 2016, Neuropsychologia.

[51]  Kazuhito Yokoi,et al.  Whole-Body Motion Generation Integrating Operator's Intention and Robot's Autonomy in Controlling Humanoid Robots , 2007, IEEE Transactions on Robotics.

[52]  Christian Mühl,et al.  EEG-based workload estimation across affective contexts , 2014, Front. Neurosci..

[53]  M. Goldberg,et al.  Saccades, salience and attention: the role of the lateral intraparietal area in visual behavior. , 2006, Progress in brain research.