A bioelectric neural interface towards intuitive prosthetic control for amputees

Objective While prosthetic hands with independently actuated digits have become commercially available, state-of-the-art human-machine interfaces (HMI) only permit control over a limited set of grasp patterns, which does not enable amputees to experience sufficient improvement in their daily activities to make an active prosthesis useful. Approach Here we present a technology platform combining fully-integrated bioelectronics, implantable intrafascicular microelectrodes and deep learning-based artificial intelligence (AI) to facilitate this missing bridge by tapping into the intricate motor control signals of peripheral nerves. The bioelectric neural interface includes an ultra-low-noise neural recording system to sense electroneurography (ENG) signals from microelectrode arrays implanted in the residual nerves, and AI models employing the recurrent neural network (RNN) architecture to decode the subject’s motor intention. Main results A pilot human study has been carried out on a transradial amputee. We demonstrate that the information channel established by the proposed neural interface is sufficient to provide high accuracy control of a prosthetic hand up to 15 degrees of freedom (DOF). The interface is intuitive as it directly maps complex prosthesis movements to the patient’s true intention. Significance Our study layouts the foundation towards not only a robust and dexterous control strategy for modern neuroprostheses at a near-natural level approaching that of the able hand, but also an intuitive conduit for connecting human minds and machines through the peripheral neural pathways. Clinical trial DExterous Hand Control Through Fascicular Targeting (DEFT). Identifier: NCT02994160.

[1]  Gail Latlief,et al.  How do the outcomes of the DEKA Arm compare to conventional prostheses? , 2018, PloS one.

[2]  Loredana Zollo,et al.  Control of Prosthetic Hands via the Peripheral Nervous System , 2016, Front. Neurosci..

[3]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[4]  V. John Mathews,et al.  Polynomial Kalman filter for myoelectric prosthetics using efficient kernel ridge regression , 2017, 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER).

[5]  Jian Xu,et al.  A High Performance Delta-Sigma Modulator for Neurosensing , 2015, Sensors.

[6]  O. Stavdahl,et al.  Control of Upper Limb Prostheses: Terminology and Proportional Myoelectric Control—A Review , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[9]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[10]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[11]  Aidan D. Roche,et al.  Prosthetic Myoelectric Control Strategies: A Clinical Perspective , 2014, Current Surgery Reports.

[12]  Alin Albu-Schäffer,et al.  The DLR hand arm system , 2011, 2011 IEEE International Conference on Robotics and Automation.

[13]  Stuart D. Harshbarger,et al.  An Overview of the Developmental Process for the Modular Prosthetic Limb , 2011 .

[14]  Hongwei Mao,et al.  Decoding arm speed during reaching , 2018, Nature Communications.

[15]  Linda Resnik,et al.  User and clinician perspectives on DEKA arm: results of VA study to optimize DEKA arm. , 2014, Journal of rehabilitation research and development.

[16]  G. Lundborg,et al.  Refined myoelectric control in below-elbow amputees using artificial neural networks and a data glove. , 2005, The Journal of hand surgery.

[17]  Hugh M. Herr,et al.  Split-belt adaptation and gait symmetry in transtibial amputees walking with a hybrid EMG controlled ankle-foot prosthesis , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[18]  Gregory A. Clark,et al.  Improved Training Paradigms and Motor-decode Algorithms: Results from Intact Individuals and a Recent Transradial Amputee with Prior Complex Regional Pain Syndrome , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[19]  Andrew E. Deczky Data acquisition systems , 1983 .

[20]  Jian Xu,et al.  A programmable fully-integrated microstimulator for neural implants and instrumentation , 2016, 2016 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[21]  Keum Shik Hong,et al.  Selectivity and Longevity of Peripheral-Nerve and Machine Interfaces: A Review , 2017, Front. Neurorobot..

[22]  John Lachapelle,et al.  Dexterous Hand Control Through Fascicular Targeting (HAPTIX-DEFT): Level 4 Evidence , 2017 .

[23]  David J. Warren,et al.  Recording and Decoding for Neural Prostheses , 2016, Proceedings of the IEEE.

[24]  Jian Xu,et al.  A 14-bit 0.17mm2 SAR ADC in 0.13μm CMOS for high precision nerve recording , 2015, 2015 IEEE Custom Integrated Circuits Conference (CICC).

[25]  Dario Farina,et al.  EMG-based simultaneous and proportional estimation of wrist/hand kinematics in uni-lateral trans-radial amputees , 2011, Journal of NeuroEngineering and Rehabilitation.

[26]  Edward Keefer,et al.  Fascicle specific targeting for selective peripheral nerve stimulation , 2019, Journal of neural engineering.

[27]  Z T Irwin,et al.  Chronic recording of hand prosthesis control signals via a regenerative peripheral nerve interface in a rhesus macaque. , 2016, Journal of neural engineering.

[28]  Jian Xu,et al.  A 13µW 87dB dynamic range implantable ΔΣ modulator for full-spectrum neural recording , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[29]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[30]  Wenfeng Zhao,et al.  A Wide Dynamic Range Neural Data Acquisition System With High-Precision Delta-Sigma ADC and On-Chip EC-PC Spike Processor , 2020, IEEE Transactions on Biomedical Circuits and Systems.

[31]  Cynthia A. Chestek,et al.  Design and testing of a 96-channel neural interface module for the Networked Neuroprosthesis system , 2019, Bioelectronic Medicine.

[32]  V. John Mathews,et al.  Individual hand movement detection and classification using peripheral nerve signals , 2017, 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER).

[33]  Jian Xu,et al.  A New System Architecture for Future Long-Term High-Density Neural Recording , 2013, IEEE Transactions on Circuits and Systems II: Express Briefs.

[34]  Linda Resnik,et al.  The DEKA Arm: Its features, functionality, and evolution during the Veterans Affairs Study to optimize the DEKA Arm , 2014, Prosthetics and orthotics international.

[35]  Cynthia A. Chestek,et al.  Closed-Loop Continuous Hand Control via Chronic Recording of Regenerative Peripheral Nerve Interfaces , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[36]  Jian Xu,et al.  Neuronix enables continuous, simultaneous neural recording and electrical microstimulation , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[37]  Linda Resnik,et al.  Home Use of a Neural-connected Sensory Prosthesis Provides the Functional and Psychosocial Experience of Having a Hand Again , 2018, Scientific Reports.

[38]  T. Kuiken,et al.  Myoelectric Pattern Recognition Outperforms Direct Control for Transhumeral Amputees with Targeted Muscle Reinnervation: A Randomized Clinical Trial , 2017, Scientific Reports.

[39]  Jian Xu,et al.  A 50 μW/Ch artifacts-insensitive neural recorder using frequency-shaping technique , 2013, Proceedings of the IEEE 2013 Custom Integrated Circuits Conference.

[40]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[41]  Jian Xu,et al.  A Bidirectional Neuromodulation Technology for Nerve Recording and Stimulation , 2018, Micromachines.

[42]  T. Kuiken,et al.  Neural Interfaces for Control of Upper Limb Prostheses: The State of the Art and Future Possibilities , 2011, PM & R : the journal of injury, function, and rehabilitation.

[43]  Silvestro Micera,et al.  A critical review of interfaces with the peripheral nervous system for the control of neuroprostheses and hybrid bionic systems , 2005, Journal of the peripheral nervous system : JPNS.

[44]  David J. Warren,et al.  Restoration of motor control and proprioceptive and cutaneous sensation in humans with prior upper-limb amputation via multiple Utah Slanted Electrode Arrays (USEAs) implanted in residual peripheral arm nerves , 2017, Journal of NeuroEngineering and Rehabilitation.

[45]  Todd A Kuiken,et al.  Targeted Muscle Reinnervation and Advanced Prosthetic Arms , 2015, Seminars in Plastic Surgery.

[46]  Vikash Kumar,et al.  MuJoCo HAPTIX: A virtual reality system for hand manipulation , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[47]  Loredana Zollo,et al.  Literature Review on Needs of Upper Limb Prosthesis Users , 2016, Front. Neurosci..

[48]  Z T Irwin,et al.  Neural control of finger movement via intracortical brain–machine interface , 2017, Journal of neural engineering.

[49]  Roland Siegwart,et al.  The hand of the DLR Hand Arm System: Designed for interaction , 2012, Int. J. Robotics Res..

[50]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[51]  Wojciech Zaremba,et al.  Recurrent Neural Network Regularization , 2014, ArXiv.

[52]  John P. Donoghue,et al.  Decoding 3-D Reach and Grasp Kinematics From High-Frequency Local Field Potentials in Primate Primary Motor Cortex , 2010, IEEE Transactions on Biomedical Engineering.

[53]  Dario Farina,et al.  The Extraction of Neural Information from the Surface EMG for the Control of Upper-Limb Prostheses: Emerging Avenues and Challenges , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[54]  Erik Scheme,et al.  Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. , 2011, Journal of rehabilitation research and development.

[55]  D J Tyler,et al.  Evaluation of high-density, multi-contact nerve cuffs for activation of grasp muscles in monkeys , 2018, Journal of neural engineering.

[56]  Jian Xu,et al.  A Frequency Shaping Neural Recorder With 3 pF Input Capacitance and 11 Plus 4.5 Bits Dynamic Range , 2014, IEEE Transactions on Biomedical Circuits and Systems.

[57]  Jian Xu,et al.  Advances in Neural Recording and Stimulation Devices , 2019, Engineering in Medicine.

[58]  G A Clark,et al.  Restoring motor control and sensory feedback in people with upper extremity amputations using arrays of 96 microelectrodes implanted in the median and ulnar nerves , 2016, Journal of neural engineering.

[59]  Wenfeng Zhao,et al.  A Low-Noise, Wireless, Frequency-Shaping Neural Recorder , 2018, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[60]  Samy Bengio,et al.  Order Matters: Sequence to sequence for sets , 2015, ICLR.

[61]  D. Farina,et al.  Linear and Nonlinear Regression Techniques for Simultaneous and Proportional Myoelectric Control , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[62]  G.S. Dhillon,et al.  Direct neural sensory feedback and control of a prosthetic arm , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[63]  S Micera,et al.  Control of Hand Prostheses Using Peripheral Information , 2010, IEEE Reviews in Biomedical Engineering.

[64]  P. Rossini,et al.  Double nerve intraneural interface implant on a human amputee for robotic hand control , 2010, Clinical Neurophysiology.

[65]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[66]  Andre Esteva,et al.  A guide to deep learning in healthcare , 2019, Nature Medicine.