A Simple ERP Method for Quantitative Analysis of Cognitive Workload in Myoelectric Prosthesis Control and Human-Machine Interaction

Common goals in the development of human-machine interface (HMI) technology are to reduce cognitive workload and increase function. However, objective and quantitative outcome measures assessing cognitive workload have not been standardized for HMI research. The present study examines the efficacy of a simple event-related potential (ERP) measure of cortical effort during myoelectric control of a virtual limb for use as an outcome tool. Participants trained and tested on two methods of control, direct control (DC) and pattern recognition control (PRC), while electroencephalographic (EEG) activity was recorded. Eighteen healthy participants with intact limbs were tested using DC and PRC under three conditions: passive viewing, easy, and hard. Novel auditory probes were presented at random intervals during testing, and significant task-difficulty effects were observed in the P200, P300, and a late positive potential (LPP), supporting the efficacy of ERPs as a cognitive workload measure in HMI tasks. LPP amplitude distinguished DC from PRC in the hard condition with higher amplitude in PRC, consistent with lower cognitive workload in PRC relative to DC for complex movements. Participants completed trials faster in the easy condition using DC relative to PRC, but completed trials more slowly using DC relative to PRC in the hard condition. The results provide promising support for ERPs as an outcome measure for cognitive workload in HMI research such as prosthetics, exoskeletons, and other assistive devices, and can be used to evaluate and guide new technologies for more intuitive HMI control.

[1]  E. Donchin,et al.  Performance of concurrent tasks: a psychophysiological analysis of the reciprocity of information-processing resources. , 1983, Science.

[2]  A. Schwartz,et al.  High-performance neuroprosthetic control by an individual with tetraplegia , 2013, The Lancet.

[3]  J. Cacioppo,et al.  Handbook Of Psychophysiology , 2019 .

[4]  D. Oleynikov,et al.  Review of surgical robotics user interface: what is the best way to control robotic surgery? , 2012, Surgical Endoscopy.

[5]  Heiner Deubel,et al.  The mind's eye : cognitive and applied aspects of eye movement research , 2003 .

[6]  Amy A Kruse,et al.  Operational neuroscience: neurophysiological measures in applied environments. , 2007, Aviation, space, and environmental medicine.

[7]  T. Kuiken,et al.  Vibrotactile detection thresholds for chest skin of amputees following targeted reinnervation surgery , 2009, Brain Research.

[8]  C D Wickens,et al.  Processing of stimulus properties: evidence for dual-task integrality. , 1985, Journal of experimental psychology. Human perception and performance.

[9]  Glenn F. Wilson,et al.  Real-Time Assessment of Mental Workload Using Psychophysiological Measures and Artificial Neural Networks , 2003, Hum. Factors.

[10]  Santosh Mathan,et al.  Towards an index of cognitive efficacy EEG-based estimation of cognitive load among individuals experiencing cancer-related cognitive decline , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[11]  Scott Makeig,et al.  Estimation of task workload from EEG data: New and current tools and perspectives , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[12]  Jose L Pons,et al.  Rehabilitation Exoskeletal Robotics , 2010, IEEE Engineering in Medicine and Biology Magazine.

[13]  J. Polich.  Updating P300: An integrative theory of P3a and P3b , 2007, Clinical Neurophysiology.

[14]  Dan Foti,et al.  Motivated and controlled attention to emotion: Time-course of the late positive potential , 2009, Clinical Neurophysiology.

[15]  Marcos Louredo,et al.  A haptic pedal for surgery assistance , 2014, Comput. Methods Programs Biomed..

[16]  Helen Y N Lindner,et al.  Upper Limb Prosthetic Outcome Measures: Review and Content Comparison Based on International Classification of Functioning, Disability and Health , 2010, Prosthetics and orthotics international.

[17]  Jacob L. Segil,et al.  Mechanical design and performance specifications of anthropomorphic prosthetic hands: a review. , 2013, Journal of rehabilitation research and development.

[18]  Rodolphe J. Gentili,et al.  Brain biomarkers of motor adaptation using phase synchronization , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[19]  Caleb J. Behrend,et al.  Update on advances in upper extremity prosthetics. , 2011, The Journal of hand surgery.

[20]  Kara D. Federmeier,et al.  Handbook of Psychophysiology: Event-Related Brain Potentials: Methods, Theory, and Applications , 2007 .

[21]  Sonja A. Kotz,et al.  Concreteness in emotional words: ERP evidence from a hemifield study , 2007, Brain Research.

[22]  Sandra Marshall,et al.  EEG and eye-tracking based measures for enhanced training , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[23]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[24]  Ivan Volosyak,et al.  An SSVEP-Based Brain–Computer Interface for the Control of Functional Electrical Stimulation , 2010, IEEE Transactions on Biomedical Engineering.

[25]  C D Wickens,et al.  Assessment of pilot performance and mental workload in rotary wing aircraft. , 1993, Ergonomics.

[26]  Peter H Veltink,et al.  Hand-opening feedback for myoelectric forearm prostheses: performance in virtual grasping tasks influenced by different levels of distraction. , 2012, Journal of rehabilitation research and development.

[27]  S Hesse,et al.  Use of the robot assisted gait therapy in rehabilitation of patients with stroke and spinal cord injury. , 2012, European journal of physical and rehabilitation medicine.

[28]  An H. Do,et al.  Brain-Computer Interface Controlled Functional Electrical Stimulation System for Ankle Movement , 2011, Journal of NeuroEngineering and Rehabilitation.

[29]  M. Cook,et al.  The utility of ambulatory electroencephalography in routine clinical practice: A critical review , 2013, Epilepsy Research.

[30]  P. Chappell,et al.  A review of clinical upper limb assessments within the framework of the WHO ICF. , 2007, Musculoskeletal care.

[31]  Farbod Fahimi,et al.  Myoelectric training systems , 2011, Expert review of medical devices.

[32]  T. Kuiken,et al.  Sensory capacity of reinnervated skin after redirection of amputated upper limb nerves to the chest , 2009, Brain : a journal of neurology.

[33]  Goro Obinata,et al.  Estimation of mental workload using saccadic eye movements in a free-viewing task , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[34]  An H. Do,et al.  Operation of a brain-computer interface walking simulator for individuals with spinal cord injury , 2013, Journal of NeuroEngineering and Rehabilitation.

[35]  Darryl G. Humphrey,et al.  Assessment of mental workload with task-irrelevant auditory probes , 1995, Biological Psychology.

[36]  Todd C. Handy,et al.  Event-related potentials : a methods handbook , 2005 .

[37]  Francisco Velasco-Álvarez,et al.  First study towards linear control of an upper-limb neuroprosthesis with an EEG-based Brain-Computer Interface , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[38]  E. Sirevaag,et al.  A Psychophysiological Assessment of Operator Workload During Simulated Flight Missions , 1987, Human factors.

[39]  Levi J. Hargrove,et al.  Classification of Simultaneous Movements Using Surface EMG Pattern Recognition , 2013, IEEE Transactions on Biomedical Engineering.

[40]  Sonja A. Kotz,et al.  Valence, arousal, and task effects in emotional prosody processing , 2013, Front. Psychol..

[41]  Todd A Kuiken,et al.  Target Achievement Control Test: evaluating real-time myoelectric pattern-recognition control of multifunctional upper-limb prostheses. , 2011, Journal of rehabilitation research and development.

[42]  Misha Pavel,et al.  Channel Selection and Feature Projection for Cognitive Load Estimation Using Ambulatory EEG , 2007, Comput. Intell. Neurosci..

[43]  Robert D. Lipschutz,et al.  Targeted muscle reinnervation for real-time myoelectric control of multifunction artificial arms. , 2009, JAMA.

[44]  Matthew W. Miller,et al.  A novel approach to the physiological measurement of mental workload. , 2011, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[45]  R Goldstein,et al.  Effects of Information-Processing Demands on Physiological Response Patterns , 1987, Human factors.

[46]  H. Jasper,et al.  The ten-twenty electrode system of the International Federation. The International Federation of Clinical Neurophysiology. , 1999, Electroencephalography and clinical neurophysiology. Supplement.

[47]  J. Johnstone,et al.  Probe evoked potentials: theory, method and applications. , 1984, The International journal of neuroscience.

[48]  U. Erdmann,et al.  Auditory probe sensitivity to mental workload changes - an event-related potential study. , 2001, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[49]  Brendan Z. Allison,et al.  Workload assessment of computer gaming using a single-stimulus event-related potential paradigm , 2008, Biological Psychology.

[50]  L. Resnik,et al.  Advanced upper limb prosthetic devices: implications for upper limb prosthetic rehabilitation. , 2012, Archives of physical medicine and rehabilitation.

[51]  José Luis Contreras-Vidal,et al.  Decoding the evolving grasping gesture from electroencephalographic (EEG) activity , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[52]  Mohammad M D Sobuh,et al.  Visuomotor behaviours when using a myoelectric prosthesis , 2014, Journal of NeuroEngineering and Rehabilitation.

[53]  Prithima R Mosaly,et al.  Subjective and objective quantification of physician's workload and performance during radiation therapy planning tasks. , 2013, Practical radiation oncology.

[54]  Zoran Nenadic,et al.  Brain-computer interface controlled functional electrical stimulation device for foot drop due to stroke , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[55]  R Parasuraman,et al.  Brain events underlying detection and recognition of weak sensory signals. , 1980, Science.

[56]  K. Crowley,et al.  A review of the evidence for P2 being an independent component process: age, sleep and modality , 2004, Clinical Neurophysiology.

[57]  Rodolphe J. Gentili,et al.  Cerebral-cortical networking and activation increase as a function of cognitive-motor task difficulty , 2012, Biological Psychology.

[58]  Atilla Kilicarslan,et al.  High accuracy decoding of user intentions using EEG to control a lower-body exoskeleton , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[59]  M. Fabiani,et al.  Naming norms for brief environmental sounds: effects of age and dementia. , 1996, Psychophysiology.

[60]  B Hudgins,et al.  Myoelectric signal processing for control of powered limb prostheses. , 2006, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[61]  Øyvind Stavdahl,et al.  Upper Limb Prosthetic Outcome Measures (ULPOM): A Working Group and Their Findings , 2009 .

[62]  Keehoon Kim,et al.  Robotic touch shifts perception of embodiment to a prosthesis in targeted reinnervation amputees. , 2011, Brain : a journal of neurology.

[63]  E. Donchin,et al.  P300 and tracking difficulty: evidence for multiple resources in dual-task performance. , 1980, Psychophysiology.