MyoTrack: Realtime Estimation of Subject Participation in Robotic Rehabilitation Using sEMG and IMU

In this paper, we present MyoTrack, a realtime classification procedure that identifies levels of subject participation during robot-assisted rehabilitation using a wearable surface electromyography (sEMG) Myo armband. We hypothesize, test, and prove that high sEMG correlates with high participation and vice versa. We then use Myo’s inertial measurement unit to extract the subject’s hand trajectory during the rehabilitation task. Comparing this hand trajectory with the ground-truth robot trajectory enables us to identify whether any high muscle activity corresponds to the active participation of the subject in robotic therapy or not (random gestures and motions). Since the robotic assistance implemented in this paper can autonomously complete the therapy task without any subject participation; it is crucial to identify the patient’s participation level in realtime and develop a suitable intervention strategy. Using 15 healthy subjects, we demonstrate that the proposed methodology of combining sEMG activation and robot-hand trajectory matching is a reliable indicator of subject participation with a realtime accuracy of 91.45%. We also present a realtime application that uses MyoTrack in the back-end to identify the realtime participation of a subject and intervene accordingly.

[1]  D.J. Reinkensmeyer,et al.  Web-based telerehabilitation for the upper extremity after stroke , 2002, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[2]  Jacob Benesty,et al.  Pearson Correlation Coefficient , 2009 .

[3]  Sofiane Boucenna,et al.  Evaluating the Engagement with Social Robots , 2015, International Journal of Social Robotics.

[4]  Katharina Stibrant Sunnerhagen,et al.  Virtual Rehabilitation after Stroke , 2008, MIE.

[5]  Thenkurussi Kesavadas,et al.  MyoTrack: Tracking Subject Participation in Robotic Rehabilitation using sEMG and IMU* , 2019, 2019 International Symposium on Medical Robotics (ISMR).

[6]  Thorsten O. Zander,et al.  Enhancing Human-Computer Interaction with Input from Active and Passive Brain-Computer Interfaces , 2010, Brain-Computer Interfaces.

[7]  Nurhazimah Nazmi,et al.  A Review of Classification Techniques of EMG Signals during Isotonic and Isometric Contractions , 2016, Sensors.

[8]  Mark Halaki,et al.  Normalization of EMG Signals: To Normalize or Not to Normalize and What to Normalize to? , 2012 .

[9]  Michelle N. Lumicao,et al.  EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. , 2007, Aviation, space, and environmental medicine.

[10]  M. Csíkszentmihályi,et al.  The Concept of Flow , 2014 .

[11]  P. Langhorne,et al.  Stroke rehabilitation , 2011, The Lancet.

[12]  Meinard Müller,et al.  Dynamic Time Warping , 2008 .

[13]  Levent Burak Kara,et al.  An image-based, trainable symbol recognizer for hand-drawn sketches , 2005, Comput. Graph..

[14]  Pramod Chembrammel,et al.  Position Error-Based Identification of Subject Participation in Robotic-Rehabilitation , 2018, 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob).

[15]  He Wang,et al.  I am a Smartwatch and I can Track my User's Arm , 2016, MobiSys.

[16]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Alexander Toet,et al.  Engagement and EMG in Serious Gaming: Experimenting with Sound and Dynamics in the Levee Patroller Training Game , 2008, Fun and Games.

[18]  Kostas Karpouzis,et al.  Investigating shared attention with a virtual agent using a gaze-based interface , 2010, Journal on Multimodal User Interfaces.

[19]  Anton Nijholt,et al.  Eye gaze patterns in conversations: there is more to conversational agents than meets the eyes , 2001, CHI.

[20]  A. Flammini,et al.  IMU-based solution for automatic detection and classification of exercises in the fitness scenario , 2017, 2017 IEEE Sensors Applications Symposium (SAS).

[21]  Pramod Chembrammel,et al.  Adaptation of Rehabilitation System Based on User’s Mental Engagement , 2015 .

[22]  Carlo J. De Luca,et al.  The Use of Surface Electromyography in Biomechanics , 1997 .

[23]  Ehsan T. Esfahani,et al.  Human Factor Study in Gesture Based CAD Environment , 2015 .

[24]  John R. Hershey,et al.  Approximating the Kullback Leibler Divergence Between Gaussian Mixture Models , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[25]  Sheeba Rosewilliam,et al.  A systematic review and synthesis of the quantitative and qualitative evidence behind patient-centred goal setting in stroke rehabilitation , 2011, Clinical rehabilitation.

[26]  Christian Kothe,et al.  Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general , 2011, Journal of neural engineering.

[27]  Alex Mihailidis,et al.  Development of a portable robot and graphical user interface for haptic rehabilitation exercise , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[28]  Richard L. Hazlett,et al.  Incorporating Facial EMG Emotion Measures as Feedback in the Software Design Process , 2005 .

[29]  Chris Berka,et al.  Real-Time Analysis of EEG Indexes of Alertness, Cognition, and Memory Acquired With a Wireless EEG Headset , 2004, Int. J. Hum. Comput. Interact..

[30]  Anthony G. Pipe,et al.  Perception of own and robot engagement in human-robot interactions and their dependence on robotics knowledge , 2014, Robotics Auton. Syst..

[31]  Pramod Chembrammel,et al.  Development and evaluation of haptics-based rehabilitation system , 2018, 2018 International Symposium on Medical Robotics (ISMR).

[32]  He Wang,et al.  MoLe: Motion Leaks through Smartwatch Sensors , 2015, MobiCom.

[33]  D. Reinkensmeyer,et al.  Effect of visual distraction and auditory feedback on patient effort during robot-assisted movement training after stroke , 2011, Journal of NeuroEngineering and Rehabilitation.

[34]  Joseph Tiran,et al.  The Jerusalem TeleRehabilitation System, a New Low-Cost, Haptic Rehabilitation Approach , 2006, Cyberpsychology Behav. Soc. Netw..

[35]  Loïc Kessous,et al.  Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis , 2010, Journal on Multimodal User Interfaces.

[36]  Robert Riener,et al.  Increasing patient engagement during virtual reality-based motor rehabilitation. , 2013, Archives of physical medicine and rehabilitation.

[37]  Evangelos Kalogerakis,et al.  RisQ: recognizing smoking gestures with inertial sensors on a wristband , 2014, MobiSys.

[38]  R.N. Scott,et al.  A new strategy for multifunction myoelectric control , 1993, IEEE Transactions on Biomedical Engineering.

[39]  Qianli Xu,et al.  Designing engagement-aware agents for multiparty conversations , 2013, CHI.

[40]  Alex Mihailidis,et al.  A haptic-robotic platform for upper-limb reaching stroke therapy: Preliminary design and evaluation results , 2007, Journal of NeuroEngineering and Rehabilitation.

[41]  J. Cram,et al.  Introduction to Surface Electromyography , 1998 .

[42]  Muhammad Imran Malik,et al.  AirScript - Creating Documents in Air , 2017, 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR).

[43]  Grant D. Huang,et al.  Robot-assisted therapy for long-term upper-limb impairment after stroke. , 2010, The New England journal of medicine.

[44]  Ana Paiva,et al.  Automatic analysis of affective postures and body motion to detect engagement with a game companion , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[45]  Abdulmotaleb El-Saddik,et al.  Haptic Virtual Rehabilitation Exercises for Poststroke Diagnosis , 2008, IEEE Transactions on Instrumentation and Measurement.

[46]  Yukiko I. Nakano,et al.  Estimating user's engagement from eye-gaze behaviors in human-agent conversations , 2010, IUI '10.