A hybrid Body-Machine Interface integrating signals from muscles and motions

OBJECTIVE Body-Machine Interfaces (BoMIs) establish a way to operate a variety of devices, allowing their users to extend the limits of their motor abilities by exploiting the redundancy of muscles and motions that remain available after spinal cord injury or stroke. Here, we considered the integration of two types of signals, motion signals derived from inertial measurement units (IMUs) and muscle activities recorded with electromyography (EMG), both contributing to the operation of the BoMI. APPROACH A direct combination of IMU and EMG signals might result in inefficient control due to the differences in their nature. Accordingly, we used a nonlinear-regression-based approach to predict IMU from EMG signals, after which the predicted and actual IMU signals were combined into a hybrid control signal. The goal of this approach was to provide users with the possibility to switch seamlessly between movement and EMG control, using the BoMI as a tool for promoting the engagement of selected muscles. We tested the interface in three control modalities, EMG-only, IMU-only and hybrid, in a cohort of 15 unimpaired participants. Participants practiced reaching movements by guiding a computer cursor over a set of targets. MAIN RESULTS We found that the proposed hybrid control led to comparable performance to IMU-based control and significantly outperformed the EMG-only control. Results also indicated that hybrid cursor control was predominantly influenced by EMG signals. SIGNIFICANCE We concluded that combining EMG with IMU signals could be an efficient way to target muscle activations while overcoming the limitations of an EMG-only control.

[1]  A. Hicks,et al.  Spasticity after spinal cord injury , 2005, Spinal Cord.

[2]  Marc W Slutzky,et al.  Reducing Abnormal Muscle Coactivation After Stroke Using a Myoelectric-Computer Interface , 2014, Neurorehabilitation and neural repair.

[3]  W. Härdle,et al.  Kernel regression smoothing of time series , 1992 .

[4]  Silvestro Micera,et al.  Effects of early and intensive neuro-rehabilitative treatment on muscle synergies in acute post-stroke patients: a pilot study , 2013, Journal of NeuroEngineering and Rehabilitation.

[5]  Elias B. Thorp,et al.  Remapping residual coordination for controlling assistive devices and recovering motor functions , 2015, Neuropsychologia.

[6]  Elisabetta Quinland,et al.  Changes in neuromuscular activity during motor training with a body-machine interface after spinal cord injury , 2017, 2017 International Conference on Rehabilitation Robotics (ICORR).

[7]  Ferdinando A. Mussa-Ivaldi,et al.  A body-machine interface for training selective pelvis movements in stroke survivors: A pilot study , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[8]  Brenna Argall,et al.  Assistive robotic manipulation through shared autonomy and a Body-Machine Interface , 2015, 2015 IEEE International Conference on Rehabilitation Robotics (ICORR).

[9]  B. Brucker,et al.  Biofeedback effect on electromyography responses in patients with spinal cord injury. , 1996, Archives of physical medicine and rehabilitation.

[10]  Elisabetta Quinland,et al.  A Myoelectric Computer Interface for Reducing Abnormal Muscle Activations after Spinal Cord Injury , 2019, 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR).

[11]  Timothy Forbes.  Mouse HCI Through Combined EMG and IMU , 2013 .

[12]  J. G. Hincapie,et al.  EMG-based Control for a C5/C6 Spinal Cord Injury Upper Extremity Neuroprosthesis , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[13]  P.E. Crago,et al.  Reciprocal EMG control of elbow extension by FES , 2001, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[14]  Lijun Zhao,et al.  Development of a hybrid motion capture method using MYO armband with application to teleoperation , 2016, 2016 IEEE International Conference on Mechatronics and Automation.

[15]  Tanja Schultz,et al.  Recognizing Hand and Finger Gestures with IMU based Motion and EMG based Muscle Activity Sensing , 2015, BIOSIGNALS.

[16]  Sethu Vijayakumar,et al.  Evaluation of regression methods for the continuous decoding of finger movement from surface EMG and accelerometry , 2015, 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER).

[17]  Nikolaos G. Tsagarakis,et al.  Tele-impedance: Teleoperation with impedance regulation using a body–machine interface , 2012, Int. J. Robotics Res..

[18]  Camilla Pierella,et al.  Upper Body-Based Power Wheelchair Control Interface for Individuals With Tetraplegia , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[19]  S J Spaulding,et al.  Muscle activity in the spinal cord-injured during wheelchair ambulation. , 1986, The American journal of occupational therapy : official publication of the American Occupational Therapy Association.

[20]  Rajiv Ranganathan,et al.  The Body-Machine Interface: A New Perspective on an Old Theme , 2012, Journal of motor behavior.

[21]  Eric Berton,et al.  Increased antagonist muscle activity in cervical SCI patients suggests altered reciprocal inhibition during elbow contractions , 2016, Clinical Neurophysiology.

[22]  M. Leathley,et al.  Prevalence of spasticity post stroke , 2002, Clinical rehabilitation.

[23]  Roozbeh Jafari,et al.  A Wearable System for Recognizing American Sign Language in Real-Time Using IMU and Surface EMG Sensors , 2016, IEEE Journal of Biomedical and Health Informatics.

[24]  Oishee Mazumder,et al.  Hand Gesture Recognition Based Omnidirectional Wheelchair Control Using IMU and EMG Sensors , 2018, J. Intell. Robotic Syst..

[25]  Ferdinando A. Mussa-Ivaldi,et al.  A body machine interface based on inertial sensors , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[26]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[27]  Gregor Schöner,et al.  The uncontrolled manifold concept: identifying control variables for a functional task , 1999, Experimental Brain Research.

[28]  Rajiv Ranganathan,et al.  Body-Machine Interface Enables People With Cervical Spinal Cord Injury to Control Devices With Available Body Movements: Proof of Concept , 2017, Neurorehabilitation and neural repair.

[29]  Loredana Zollo,et al.  Fusion of M-IMU and EMG signals for the control of trans-humeral prostheses , 2016, 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[30]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[31]  Matteo Bianchi,et al.  Assessment of Myoelectric Controller Performance and Kinematic Behavior of a Novel Soft Synergy-Inspired Robotic Hand for Prosthetic Applications , 2016, Front. Neurorobot..

[32]  Guangjun Liu,et al.  A novel HCI based on EMG and IMU , 2011, 2011 IEEE International Conference on Robotics and Biomimetics.

[33]  John McPhee,et al.  On the Relationship Between Muscle Synergies and Redundant Degrees of Freedom in Musculoskeletal Systems , 2019, Front. Comput. Neurosci..

[34]  F. Mussa-Ivaldi,et al.  Functional reorganization of upper-body movement after spinal cord injury , 2010, Experimental Brain Research.

[35]  Ferdinando A. Mussa-Ivaldi,et al.  Body-Machine Interfaces after Spinal Cord Injury: Rehabilitation and Brain Plasticity , 2016, Brain sciences.

[36]  Bernhard Schölkopf,et al.  Kernel Principal Component Analysis , 1997, ICANN.

[37]  Jacob L Segil,et al.  Novel postural control algorithm for control of multifunctional myoelectric prosthetic hands. , 2015, Journal of rehabilitation research and development.

[38]  Silvestro Micera,et al.  Data-driven body–machine interface for the accurate control of drones , 2018, Proceedings of the National Academy of Sciences.

[39]  M. Kramer.  Nonlinear principal component analysis using autoassociative neural networks , 1991 .

[40]  Dario Farina,et al.  Online mapping of EMG signals into kinematics by autoencoding , 2018, Journal of NeuroEngineering and Rehabilitation.

[41]  Michael I. Jordan,et al.  Optimal feedback control as a theory of motor coordination , 2002, Nature Neuroscience.

[42]  Reza Shadmehr,et al.  Quantifying Generalization from Trial-by-Trial Behavior of Adaptive Systems that Learn with Basis Functions: Theory and Experiments in Human Motor Control , 2003, The Journal of Neuroscience.

[43]  G. Gini,et al.  An EMG-controlled exoskeleton for hand rehabilitation , 2005, 9th International Conference on Rehabilitation Robotics, 2005. ICORR 2005..

[44]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[45]  Maura Casadio,et al.  Sensory motor remapping of space in human-machine interfaces. , 2011, Progress in brain research.

[46]  Maura Casadio,et al.  The dynamics of motor learning through the formation of internal models , 2019, PLoS computational biology.

[47]  D. Farina,et al.  Linear and Nonlinear Regression Techniques for Simultaneous and Proportional Myoelectric Control , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.