A bioelectric neural interface towards intuitive prosthetic control for amputees.

OBJECTIVE While prosthetic hands with independently actuated digits have become commercially available, state-of-the-art human-machine interfaces (HMI) only permit control over a limited set of grasp patterns, which does not enable amputees to experience sufficient improvement in their daily activities to make an active prosthesis useful. APPROACH Here we present a technology platform combining fully-integrated bioelectronics, implantable intrafascicular microelectrodes and deep learning-based artificial intelligence (AI) to facilitate this missing bridge by tapping into the intricate motor control signals of peripheral nerves. The bioelectric neural interface includes an ultra-low-noise neural recording system to sense electroneurography (ENG) signals from microelectrode arrays implanted in the residual nerves, and AI models employing the recurrent neural network (RNN) architecture to decode the subject's motor intention. MAIN RESULTS A pilot human study has been carried out on a transradial amputee. We demonstrate that the information channel established by the proposed neural interface is sufficient to provide high accuracy control of a prosthetic hand up to 15 degrees of freedom (DOF). The interface is intuitive as it directly maps complex prosthesis movements to the patient's true intention. SIGNIFICANCE Our study layouts the foundation towards not only a robust and dexterous control strategy for modern neuroprostheses at a near-natural level approaching that of the able hand, but also an intuitive conduit for connecting human minds and machines through the peripheral neural pathways. (Clinical trial identifier: NCT02994160).

[1]  Cynthia A. Chestek,et al.  Closed-Loop Continuous Hand Control via Chronic Recording of Regenerative Peripheral Nerve Interfaces , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[2]  Jian Xu,et al.  Neuronix enables continuous, simultaneous neural recording and electrical microstimulation , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[3]  David J. Warren,et al.  Restoration of motor control and proprioceptive and cutaneous sensation in humans with prior upper-limb amputation via multiple Utah Slanted Electrode Arrays (USEAs) implanted in residual peripheral arm nerves , 2017, Journal of NeuroEngineering and Rehabilitation.

[4]  Z T Irwin,et al.  Chronic recording of hand prosthesis control signals via a regenerative peripheral nerve interface in a rhesus macaque. , 2016, Journal of neural engineering.

[5]  Gail Latlief,et al.  How do the outcomes of the DEKA Arm compare to conventional prostheses? , 2018, PloS one.

[6]  Linda Resnik,et al.  Home Use of a Neural-connected Sensory Prosthesis Provides the Functional and Psychosocial Experience of Having a Hand Again , 2018, Scientific Reports.

[7]  David J. Warren,et al.  Recording and Decoding for Neural Prostheses , 2016, Proceedings of the IEEE.

[8]  T. Kuiken,et al.  Myoelectric Pattern Recognition Outperforms Direct Control for Transhumeral Amputees with Targeted Muscle Reinnervation: A Randomized Clinical Trial , 2017, Scientific Reports.

[9]  John Lachapelle,et al.  Dexterous Hand Control Through Fascicular Targeting (HAPTIX-DEFT): Level 4 Evidence , 2017 .

[10]  Andre Esteva,et al.  A guide to deep learning in healthcare , 2019, Nature Medicine.

[11]  Z T Irwin,et al.  Neural control of finger movement via intracortical brain–machine interface , 2017, Journal of neural engineering.

[12]  Loren Rieth,et al.  A novel microwire interface for small diameter peripheral nerves in a chronic, awake murine model , 2020, Journal of neural engineering.

[13]  T. Kuiken,et al.  Neural Interfaces for Control of Upper Limb Prostheses: The State of the Art and Future Possibilities , 2011, PM & R : the journal of injury, function, and rehabilitation.

[14]  Roland Siegwart,et al.  The hand of the DLR Hand Arm System: Designed for interaction , 2012, Int. J. Robotics Res..

[15]  Dario Farina,et al.  The Extraction of Neural Information from the Surface EMG for the Control of Upper-Limb Prostheses: Emerging Avenues and Challenges , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[16]  Wenfeng Zhao,et al.  A Low-Noise, Wireless, Frequency-Shaping Neural Recorder , 2018, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[17]  Jian Xu,et al.  A High Performance Delta-Sigma Modulator for Neurosensing , 2015, Sensors.

[18]  O. Stavdahl,et al.  Control of Upper Limb Prostheses: Terminology and Proportional Myoelectric Control—A Review , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[19]  Jian Xu,et al.  A 50 μW/Ch artifacts-insensitive neural recorder using frequency-shaping technique , 2013, Proceedings of the IEEE 2013 Custom Integrated Circuits Conference.

[20]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[21]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Erik Scheme,et al.  Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. , 2011, Journal of rehabilitation research and development.

[23]  Jian Xu,et al.  A Bidirectional Neuromodulation Technology for Nerve Recording and Stimulation , 2018, Micromachines.

[24]  Wenfeng Zhao,et al.  A Wide Dynamic Range Neural Data Acquisition System With High-Precision Delta-Sigma ADC and On-Chip EC-PC Spike Processor , 2020, IEEE Transactions on Biomedical Circuits and Systems.

[25]  C. T. Clarke,et al.  A new method for spike extraction using velocity selective recording demonstrated with physiological ENG in Rat , 2015, Journal of Neuroscience Methods.

[26]  G A Clark,et al.  Restoring motor control and sensory feedback in people with upper extremity amputations using arrays of 96 microelectrodes implanted in the median and ulnar nerves , 2016, Journal of neural engineering.

[27]  Cynthia A. Chestek,et al.  Design and testing of a 96-channel neural interface module for the Networked Neuroprosthesis system , 2019, Bioelectronic Medicine.

[28]  V. John Mathews,et al.  Individual hand movement detection and classification using peripheral nerve signals , 2017, 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER).

[29]  D. Farina,et al.  Linear and Nonlinear Regression Techniques for Simultaneous and Proportional Myoelectric Control , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[30]  Jian Xu,et al.  A New System Architecture for Future Long-Term High-Density Neural Recording , 2013, IEEE Transactions on Circuits and Systems II: Express Briefs.

[31]  G.S. Dhillon,et al.  Direct neural sensory feedback and control of a prosthetic arm , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[32]  S Micera,et al.  Control of Hand Prostheses Using Peripheral Information , 2010, IEEE Reviews in Biomedical Engineering.

[33]  G. Lundborg,et al.  Refined myoelectric control in below-elbow amputees using artificial neural networks and a data glove. , 2005, The Journal of hand surgery.

[34]  Gregory A. Clark,et al.  Improved Training Paradigms and Motor-decode Algorithms: Results from Intact Individuals and a Recent Transradial Amputee with Prior Complex Regional Pain Syndrome , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[35]  Alin Albu-Schäffer,et al.  The DLR hand arm system , 2011, 2011 IEEE International Conference on Robotics and Automation.

[36]  Loredana Zollo,et al.  Control of Prosthetic Hands via the Peripheral Nervous System , 2016, Front. Neurosci..

[37]  Aidan D. Roche,et al.  Prosthetic Myoelectric Control Strategies: A Clinical Perspective , 2014, Current Surgery Reports.

[38]  P. Rossini,et al.  Double nerve intraneural interface implant on a human amputee for robotic hand control , 2010, Clinical Neurophysiology.

[39]  J. T. Taylor,et al.  First demonstration of velocity selective recording from the pig vagus using a nerve cuff shows respiration afferents , 2018, Biomedical engineering letters.

[40]  Linda Resnik,et al.  The DEKA Arm: Its features, functionality, and evolution during the Veterans Affairs Study to optimize the DEKA Arm , 2014, Prosthetics and orthotics international.

[41]  José Zariffa,et al.  Classification of naturally evoked compound action potentials in peripheral nerve spatiotemporal recordings , 2018, Scientific Reports.

[42]  V. John Mathews,et al.  Polynomial Kalman filter for myoelectric prosthetics using efficient kernel ridge regression , 2017, 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER).

[43]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[44]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[45]  D J Tyler,et al.  Evaluation of high-density, multi-contact nerve cuffs for activation of grasp muscles in monkeys , 2018, Journal of neural engineering.

[46]  John P. Donoghue,et al.  Decoding 3-D Reach and Grasp Kinematics From High-Frequency Local Field Potentials in Primate Primary Motor Cortex , 2010, IEEE Transactions on Biomedical Engineering.

[47]  Jian Xu,et al.  A 13µW 87dB dynamic range implantable ΔΣ modulator for full-spectrum neural recording , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[48]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[49]  Silvestro Micera,et al.  A critical review of interfaces with the peripheral nervous system for the control of neuroprostheses and hybrid bionic systems , 2005, Journal of the peripheral nervous system : JPNS.

[50]  Jian Xu,et al.  A programmable fully-integrated microstimulator for neural implants and instrumentation , 2016, 2016 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[51]  Keum Shik Hong,et al.  Selectivity and Longevity of Peripheral-Nerve and Machine Interfaces: A Review , 2017, Front. Neurorobot..

[52]  Dario Farina,et al.  EMG-based simultaneous and proportional estimation of wrist/hand kinematics in uni-lateral trans-radial amputees , 2011, Journal of NeuroEngineering and Rehabilitation.

[53]  Edward Keefer,et al.  Fascicle specific targeting for selective peripheral nerve stimulation , 2019, Journal of neural engineering.

[54]  Jian Xu,et al.  A Frequency Shaping Neural Recorder With 3 pF Input Capacitance and 11 Plus 4.5 Bits Dynamic Range , 2014, IEEE Transactions on Biomedical Circuits and Systems.

[55]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[56]  Hongwei Mao,et al.  Decoding arm speed during reaching , 2018, Nature Communications.

[57]  Linda Resnik,et al.  User and clinician perspectives on DEKA arm: results of VA study to optimize DEKA arm. , 2014, Journal of rehabilitation research and development.

[58]  Todd A Kuiken,et al.  Targeted Muscle Reinnervation and Advanced Prosthetic Arms , 2015, Seminars in Plastic Surgery.

[59]  Vikash Kumar,et al.  MuJoCo HAPTIX: A virtual reality system for hand manipulation , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[60]  Loredana Zollo,et al.  Literature Review on Needs of Upper Limb Prosthesis Users , 2016, Front. Neurosci..