Machine Learning for Motor Learning: EEG-based Continuous Assessment of Cognitive Engagement for Adaptive Rehabilitation Robots

Although cognitive engagement (CE) is crucial for motor learning, it remains underutilized in rehabilitation robots, partly because its assessment currently relies on subjective and gross measurements taken intermittently. Here, we propose an end-to-end computational framework that assesses CE in near real-time, using electroencephalography (EEG) signals as objective measurements. The framework consists of i) a deep convolutional neural network that extracts task-discriminative spatiotemporal EEG features to predict the level of CE for two classes- cognitively engaged vs. disengaged; and ii) a novel sliding window method that predicts continuous levels of CE in short time intervals. We evaluated our framework on 8 healthy subjects using an in-house Go/No-Go experiment that adapted its gameplay parameters to induce cognitive fatigue. The proposed CNN had an average leave-one-subject-out accuracy of 88.19%. The CE prediction correlated well with a commonly used behavioral metric based on self-reports taken every 5 minutes $(\rho =0.93)$. Our results objectify CE measurement in near real-time and pave the way for using CE as a rehabilitation parameter for tailoring robotic therapy to each patient’s needs and skills.

[1]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[2]  J. Patton,et al.  Evaluation of robotic training forces that either enhance or reduce error in chronic hemiparetic stroke survivors , 2005, Experimental Brain Research.

[3]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[4]  Michelle N. Lumicao,et al.  EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. , 2007, Aviation, space, and environmental medicine.

[5]  Neelesh Kumar Camera-based Detection of the Early Stages of Fatigue : Validation with MEG and Self-Assessment Data , 2017 .

[6]  Kostis P. Michmizos,et al.  Virtual reality for pediatric neuro-rehabilitation: Adaptive visual feedback of movement to engage the mirror neuron system , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[7]  Claude Frasson,et al.  Modeling mental workload using EEG features for intelligent systems , 2011, UMAP'11.

[8]  Barbara A. Greene Measuring Cognitive Engagement With Self-Report Scales: Reflections From Over 20 Years of Research , 2015 .

[9]  Narasimhan Sundararajan,et al.  Classification of Mental Tasks from Eeg Signals Using Extreme Learning Machine , 2006, Int. J. Neural Syst..

[10]  Diane L Damiano,et al.  Activity, Activity, Activity: Rethinking Our Physical Therapy Approach to Cerebral Palsy , 2006, Physical Therapy.

[11]  H. A. BEAGLEY,et al.  Objective evaluation of auditory evoked EEG responses , 1974, Nature.

[12]  Samy Bengio,et al.  Understanding deep learning requires rethinking generalization , 2016, ICLR.

[13]  J. Cacioppo,et al.  Handbook Of Psychophysiology , 2019 .

[14]  B. Volpe,et al.  Kinematic Robot-Based Evaluation Scales and Clinical Counterparts to Measure Upper Limb Motor Performance in Patients With Chronic Stroke , 2010, Neurorehabilitation and neural repair.

[15]  N. Hogan,et al.  Robot-aided neurorehabilitation. , 1998, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[16]  H. Krebs,et al.  Pediatric robotic rehabilitation: Current knowledge and future trends in treating children with sensorimotor impairments. , 2017, NeuroRehabilitation.

[17]  Martin Luessi,et al.  MEG and EEG data analysis with MNE-Python , 2013, Front. Neuroinform..

[18]  Ferdinando A. Mussa-Ivaldi,et al.  Robot-assisted adaptive training: custom force fields for teaching movement patterns , 2004, IEEE Transactions on Biomedical Engineering.

[19]  N. Hogan,et al.  Increasing productivity and quality of care: robot-aided neuro-rehabilitation. , 2000, Journal of rehabilitation research and development.

[20]  Hermano I Krebs,et al.  Robotic Measurement of Arm Movements After Stroke Establishes Biomarkers of Motor Recovery , 2014, Stroke.

[21]  Hermano Igo Krebs,et al.  Rehabilitation Robotics: Performance-Based Progressive Robot-Assisted Therapy , 2003, Auton. Robots.

[22]  Ronald H. Stevens,et al.  EEG-Related Changes in Cognitive Workload, Engagement and Distraction as Students Acquire Problem Solving Skills , 2007, User Modeling.

[23]  K. P. Michmizos,et al.  Assist-as-needed in lower extremity robotic therapy for children with cerebral palsy , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[24]  Lujo Bauer,et al.  Human-in-the-loop 에이전트 기반 모델링 및 시뮬레이션 구현 , 2014 .

[25]  Yongtian He,et al.  Deep learning for electroencephalogram (EEG) classification tasks: a review , 2019, Journal of neural engineering.

[26]  C. Braun,et al.  Motor learning elicited by voluntary drive. , 2003, Brain : a journal of neurology.

[27]  Hermano Igo Krebs,et al.  A Comparative Analysis of Speed Profile Models for Ankle Pointing Movements: Evidence that Lower and Upper Extremity Discrete Movements are Controlled by a Single Invariant Strategy , 2014, Front. Hum. Neurosci..

[28]  Hermano Igo Krebs,et al.  Robot-Aided Neurorehabilitation: A Pediatric Robot for Ankle Rehabilitation , 2015, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[29]  N. Hogan,et al.  Overview of clinical trials with MIT-MANUS: a robot-aided neuro-rehabilitation facility. , 1999, Technology and health care : official journal of the European Society for Engineering and Medicine.

[30]  N. A. Bernshteĭn The co-ordination and regulation of movements , 1967 .

[31]  Vladimir Krajca,et al.  Objective Assessment of the Degree of Dementia by Means of EEG , 2003, Neuropsychobiology.

[32]  Tong Zhang,et al.  A Novel Neural Network Model based on Cerebral Hemispheric Asymmetry for EEG Emotion Recognition , 2018, IJCAI.

[33]  Sayers Bm,et al.  Objective evaluation of auditory evoked EEG responses. , 1974 .

[34]  Natasha M. Maurits,et al.  Mental Fatigue Affects Visual Selective Attention , 2012, PloS one.

[35]  Hermano Igo Krebs,et al.  Beyond Human or Robot Administered Treadmill Training , 2012 .