High-Density Electromyography and Motor Skill Learning for Robust Long-Term Control of a 7-DoF Robot Arm

Myoelectric control offers a direct interface between human intent and various robotic applications through recorded muscle activity. Traditional control schemes realize this interface through direct mapping or pattern recognition techniques. The former approach provides reliable control at the expense of functionality, while the latter increases functionality at the expense of long-term reliability. An alternative approach, using concepts of motor learning, provides session-independent simultaneous control, but previously relied on consistent electrode placement over biomechanically independent muscles. This paper extends the functionality and practicality of the motor learning-based approach, using high-density electrode grids and muscle synergy-inspired decomposition to generate control inputs with reduced constraints on electrode placement. The method is demonstrated via real-time simultaneous and proportional control of a 4-DoF myoelectric interface over multiple days. Subjects showed learning trends consistent with typical motor skill learning without requiring any retraining or recalibration between sessions. Moreover, they adjusted to physical constraints of a robot arm after learning the control in a constraint-free virtual interface, demonstrating robust control as they performed precision tasks. The results demonstrate the efficacy of the proposed man-machine interface as a viable alternative to conventional control schemes for myoelectric interfaces designed for long-term use.

[1]  Panagiotis K. Artemiadis,et al.  An EMG-Based Robot Control Scheme Robust to Time-Varying EMG Signal Features , 2010, IEEE Transactions on Information Technology in Biomedicine.

[2]  Aymar de Rugy,et al.  Are muscle synergies useful for neural control? , 2013, Front. Comput. Neurosci..

[3]  Farbod Fahimi,et al.  Online human training of a myoelectric prosthesis controller via actor-critic reinforcement learning , 2011, 2011 IEEE International Conference on Rehabilitation Robotics.

[4]  Levi J. Hargrove,et al.  Real-time comparison of conventional direct control and pattern recognition myoelectric control in a two-dimensional Fitts' law style test , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[5]  Dingguo Zhang,et al.  Application of a self-enhancing classification method to electromyography pattern recognition for multifunctional prosthesis control , 2013, Journal of NeuroEngineering and Rehabilitation.

[6]  Levi J. Hargrove,et al.  Real-Time and Offline Performance of Pattern Recognition Myoelectric Control Using a Generic Electrode Grid With Targeted Muscle Reinnervation Patients , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[7]  Erik Scheme,et al.  Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. , 2011, Journal of rehabilitation research and development.

[8]  Imteyaz Ahmad,et al.  A Review of EMG recording technique , 2012 .

[9]  Panagiotis K. Artemiadis,et al.  EMG-Based Control of a Robot Arm Using Low-Dimensional Embeddings , 2010, IEEE Transactions on Robotics.

[10]  Panagiotis K. Artemiadis,et al.  A Switching Regime Model for the EMG-Based Control of a Robot Arm , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  J. Carmena Advances in Neuroprosthetic Learning and Control , 2013, PLoS biology.

[12]  R. Stickgold,et al.  Practice with Sleep Makes Perfect Sleep-Dependent Motor Skill Learning , 2002, Neuron.

[13]  Panagiotis Artemiadis,et al.  A hybrid BMI-based exoskeleton for paresis: EMG control for assisting arm movements , 2017, Journal of neural engineering.

[14]  Panagiotis K. Artemiadis,et al.  Proportional Myoelectric Control of Robots: Muscle Synergy Development Drives Performance Enhancement, Retainment, and Generalization , 2015, IEEE Transactions on Robotics.

[15]  Dario Farina,et al.  Intuitive, Online, Simultaneous, and Proportional Myoelectric Control Over Two Degrees-of-Freedom in Upper Limb Amputees , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[16]  Dario Farina,et al.  Is Accurate Mapping of EMG Signals on Kinematics Needed for Precise Online Myoelectric Control? , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[17]  A. Luchins Mechanization in problem solving: The effect of Einstellung. , 1942 .

[18]  K. Nazarpour,et al.  Negentropy analysis of surface electromyogram signal , 2005, IEEE/SP 13th Workshop on Statistical Signal Processing, 2005.

[19]  A.D.C. Chan,et al.  Examining the adverse effects of limb position on pattern recognition based myoelectric control , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[20]  Todd A Kuiken,et al.  Real-time simultaneous and proportional myoelectric control using intramuscular EMG , 2014, Journal of neural engineering.

[21]  Panagiotis Artemiadis,et al.  Embedded Human Control of Robots Using Myoelectric Interfaces , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[22]  Ryan Clingman,et al.  A Novel Myoelectric Training Device for Upper Limb Prostheses , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[23]  Rami N. Khushaba,et al.  Correlation Analysis of Electromyogram Signals for Multiuser Myoelectric Interfaces , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[24]  Christian Cipriani,et al.  Abstract and Proportional Myoelectric Control for Multi-Fingered Hand Prostheses , 2013, Annals of Biomedical Engineering.

[25]  Dario Farina,et al.  EMG-based simultaneous and proportional estimation of wrist/hand kinematics in uni-lateral trans-radial amputees , 2011, Journal of NeuroEngineering and Rehabilitation.

[26]  T. Kuiken,et al.  Neural Interfaces for Control of Upper Limb Prostheses: The State of the Art and Future Possibilities , 2011, PM & R : the journal of injury, function, and rehabilitation.

[27]  Ferat Sahin,et al.  Camera control with EMG signals using Principal Component Analysis and support vector machines , 2011, 2011 IEEE International Systems Conference.

[28]  D. Farina,et al.  Simultaneous and Proportional Estimation of Hand Kinematics From EMG During Mirrored Movements at Multiple Degrees-of-Freedom , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[29]  Manfredo Atzori,et al.  Electromyography data for non-invasive naturally-controlled robotic hand prostheses , 2014, Scientific Data.

[30]  Panagiotis K. Artemiadis,et al.  Learning efficient control of robots using myoelectric interfaces , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[31]  Panagiotis K. Artemiadis,et al.  Enhancing practical multifunctional myoelectric applications through implicit motor control training systems , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[32]  Aymar de Rugy,et al.  Muscle Coordination Is Habitual Rather than Optimal , 2012, The Journal of Neuroscience.

[33]  Andrew Jackson,et al.  Learning a Novel Myoelectric-Controlled Interface Task , 2008, Journal of neurophysiology.

[34]  B. Hudgins,et al.  The effect of electrode displacements on pattern recognition based myoelectric control , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[35]  Patrick van der Smagt,et al.  EMG-based teleoperation and manipulation with the DLR LWR-III , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[36]  Jun Morimoto,et al.  Bilinear Modeling of EMG Signals to Extract User-Independent Features for Multiuser Myoelectric Interface , 2013, IEEE Transactions on Biomedical Engineering.

[37]  Winnie Jensen,et al.  Introduction to Neural Engineering for Motor Rehabilitation , 2013 .

[38]  D. Farina,et al.  Linear and Nonlinear Regression Techniques for Simultaneous and Proportional Myoelectric Control , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[39]  Ning Jiang,et al.  Extracting Simultaneous and Proportional Neural Control Information for Multiple-DOF Prostheses From the Surface Electromyographic Signal , 2009, IEEE Transactions on Biomedical Engineering.

[40]  Christian Cipriani,et al.  Dexterous Control of a Prosthetic Hand Using Fine-Wire Intramuscular Electrodes in Targeted Extrinsic Muscles , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[41]  J.W. Sensinger,et al.  Adaptive Pattern Recognition of Myoelectric Signals: Exploration of Conceptual Framework and Practical Algorithms , 2009, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[42]  Xiaorong Zhang,et al.  A real-time, practical sensor fault-tolerant module for robust EMG pattern recognition , 2015, Journal of NeuroEngineering and Rehabilitation.

[43]  Junuk Chu,et al.  Wearable EMG-based HCI for Electric-Powered Wheelchair Users with Motor Disabilities , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[44]  Max Ortiz-Catalan,et al.  Real-Time and Simultaneous Control of Artificial Limbs Based on Pattern Recognition Algorithms , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[45]  Nikolaos G. Tsagarakis,et al.  Teleimpedance control of a synergy-driven anthropomorphic hand , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[46]  Dario Farina,et al.  The Extraction of Neural Information from the Surface EMG for the Control of Upper-Limb Prostheses: Emerging Avenues and Challenges , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[47]  Dario Farina,et al.  Simultaneous and proportional control of 2D wrist movements with myoelectric signals , 2012, 2012 IEEE International Workshop on Machine Learning for Signal Processing.

[48]  Erik J. Scheme,et al.  Validation of a Selective Ensemble-Based Classification Scheme for Myoelectric Control Using a Three-Dimensional Fitts' Law Test , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[49]  Panagiotis Artemiadis,et al.  Beyond User-Specificity for EMG Decoding Using Multiresolution Muscle Synergy Analysis , 2013 .

[50]  José del R. Millán,et al.  Brain-Computer Interfaces , 2020, Handbook of Clinical Neurology.

[51]  Todd A. Kuiken,et al.  A Decision-Based Velocity Ramp for Minimizing the Effect of Misclassifications During Real-Time Pattern Recognition Control , 2011, IEEE Transactions on Biomedical Engineering.

[52]  Barbara Caputo,et al.  Stable myoelectric control of a hand prosthesis using non-linear incremental learning , 2014, Front. Neurorobot..

[53]  Xinjun Sheng,et al.  Reduced Daily Recalibration of Myoelectric Prosthesis Classifiers Based on Domain Adaptation , 2016, IEEE Journal of Biomedical and Health Informatics.

[54]  Justin Bayer,et al.  Continuous robot control using surface electromyography of atrophic muscles , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[55]  Silvestro Micera,et al.  On the Shared Control of an EMG-Controlled Prosthetic Hand: Analysis of User–Prosthesis Interaction , 2008, IEEE Transactions on Robotics.

[56]  Dario Farina,et al.  Extracting Signals Robust to Electrode Number and Shift for Online Simultaneous and Proportional Myoelectric Control by Factorization Algorithms , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[57]  Panagiotis Artemiadis,et al.  User-Independent Hand Motion Classification With Electromyography , 2013 .

[58]  A. Jackson,et al.  Flexible Cortical Control of Task-Specific Muscle Synergies , 2012, The Journal of Neuroscience.

[59]  Changmok Choi,et al.  Synergy matrices to estimate fluid wrist movements by surface electromyography. , 2011, Medical engineering & physics.

[60]  I. Scott MacKenzie,et al.  Towards a standard for pointing device evaluation, perspectives on 27 years of Fitts' law research in HCI , 2004, Int. J. Hum. Comput. Stud..

[61]  Lauren H Smith,et al.  A comparison of the real-time controllability of pattern recognition to conventional myoelectric control for discrete and simultaneous movements , 2012, Journal of NeuroEngineering and Rehabilitation.

[62]  Aidan D. Roche,et al.  Prosthetic Myoelectric Control Strategies: A Clinical Perspective , 2014, Current Surgery Reports.