Decoding with limited neural data: a mixture of time-warped trajectory models for directional reaches

Neuroprosthetic devices promise to allow paralyzed patients to perform the necessary functions of everyday life. However, to allow patients to use such tools it is necessary to decode their intent from neural signals such as electromyograms (EMGs). Because these signals are noisy, state of the art decoders integrate information over time. One systematic way of doing this is by taking into account the natural evolution of the state of the body--by using a so-called trajectory model. Here we use two insights about movements to enhance our trajectory model: (1) at any given time, there is a small set of likely movement targets, potentially identified by gaze; (2) reaches are produced at varying speeds. We decoded natural reaching movements using EMGs of muscles that might be available from an individual with spinal cord injury. Target estimates found from tracking eye movements were incorporated into the trajectory model, while a mixture model accounted for the inherent uncertainty in these estimates. Warping the trajectory model in time using a continuous estimate of the reach speed enabled accurate decoding of faster reaches. We found that the choice of richer trajectory models, such as those incorporating target or speed, improves decoding particularly when there is a small number of EMGs available.

[1]  S. Solla,et al.  Toward the Restoration of Hand Use to a Paralyzed Monkey: Brain-Controlled Functional Electrical Stimulation of Forearm Muscles , 2009, PloS one.

[2]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[3]  R.N. Scott,et al.  A new strategy for multifunction myoelectric control , 1993, IEEE Transactions on Biomedical Engineering.

[4]  Teresa H. Y. Meng,et al.  Model-based neural decoding of reaching movements: a maximum likelihood approach , 2004, IEEE Transactions on Biomedical Engineering.

[5]  Michael J. Black,et al.  Modeling and decoding motor cortical activity using a switching Kalman filter , 2004, IEEE Transactions on Biomedical Engineering.

[6]  Byron M. Yu,et al.  Mixture of Trajectory Models for Neural Decoding of Goal-directed Movements a Computational Model of Craving and Obsession Decoding Visual Inputs from Multiple Neurons in the Human Temporal Lobe Encoding Contribution of Individual Retinal Ganglion Cell Responses to Velocity and Acceleration , 2008 .

[7]  Michael J. Black,et al.  Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia , 2008, Journal of neural engineering.

[8]  Bryan Buchholz,et al.  ISB recommendation on definitions of joint coordinate systems of various joints for the reporting of human joint motion--Part II: shoulder, elbow, wrist and hand. , 2005, Journal of biomechanics.

[9]  R. Johansson,et al.  Eye–Hand Coordination in Object Manipulation , 2001, The Journal of Neuroscience.

[10]  C. Neuper,et al.  Combining Brain–Computer Interfaces and Assistive Technologies: State-of-the-Art and Challenges , 2010, Front. Neurosci..

[11]  A. W. Wiegner,et al.  Clinical evaluation of the Helping Hand electromechanical arm , 1996, Proceedings of 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[12]  Panagiotis K. Artemiadis,et al.  A Switching Regime Model for the EMG-Based Control of a Robot Arm , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[13]  Brendan Z. Allison,et al.  The Hybrid BCI , 2010, Frontiers in Neuroscience.

[14]  Richard A Andersen,et al.  Decoding Trajectories from Posterior Parietal Cortex Ensembles , 2008, The Journal of Neuroscience.

[15]  P. H. Peckham,et al.  An implanted upper-extremity neuroprosthesis using myoelectric control. , 2008, The Journal of hand surgery.

[16]  Andrew S. Whitford,et al.  Cortical control of a prosthetic arm for self-feeding , 2008, Nature.

[17]  Geoffrey E. Hinton,et al.  Parameter estimation for linear dynamical systems , 1996 .

[18]  Emery N. Brown,et al.  A State-Space Analysis for Reconstruction of Goal-Directed Movements Using Neural Signals , 2006, Neural Computation.

[19]  Wei Wu,et al.  Neural Decoding of Hand Motion Using a Linear State-Space Model With Hidden States , 2009, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[20]  Dennis C. Tkach,et al.  Study of stability of time-domain features for electromyographic pattern recognition , 2010, Journal of NeuroEngineering and Rehabilitation.

[21]  Ricardo Chavarriaga,et al.  A hybrid brain–computer interface based on the fusion of electroencephalographic and electromyographic activities , 2011, Journal of neural engineering.

[22]  Emery N. Brown,et al.  A State-Space Framework for Movement Control to Dynamic Goals Through Brain-Driven Interfaces , 2007, IEEE Transactions on Biomedical Engineering.

[23]  Wei Wu,et al.  Bayesian Population Decoding of Motor Cortical Activity Using a Kalman Filter , 2006, Neural Computation.

[24]  Robert F. Kirsch,et al.  UPPER AND LOWER EXTREMITY MOTOR NEUROPROSTHESES , 2004 .

[25]  D. Simon.  Optimal State Estimation: Kalman, H Infinity, and Nonlinear Approaches , 2006 .

[26]  M J Mulcahey,et al.  Development of an upper extremity FES system for individuals with C4 tetraplegia. , 1996, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[27]  Nicholas Hatsopoulos,et al.  Decoding continuous and discrete motor behaviors using motor and premotor cortical ensembles. , 2004, Journal of neurophysiology.

[28]  J.E. Kulkarni,et al.  State-Space Decoding of Goal-Directed Movements , 2008, IEEE Signal Processing Magazine.

[29]  S.I. Ryu,et al.  Cortical Neural Prosthesis Performance Improves When Eye Position Is Monitored , 2008, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[30]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[31]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[32]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[33]  R. Nathan,et al.  Upper limb functions regained in quadriplegia: a hybrid computerized neuromuscular stimulation system. , 1990, Archives of physical medicine and rehabilitation.

[34]  Robert F. Kirsch,et al.  An Implanted Neuroprosthesis for High Tetraplegia , 2005 .

[35]  Tommy Strandvall,et al.  Eye Tracking in Human-Computer Interaction and Usability Research , 2009, INTERACT.

[36]  Konrad P. Körding,et al.  Mixture of time-warped trajectory models for movement decoding , 2010, NIPS.

[37]  J. Mcdonald,et al.  Spinal-cord injury , 2002, The Lancet.

[38]  Keith S. Karn,et al.  Commentary on Section 4. Eye tracking in human-computer interaction and usability research: Ready to deliver the promises. , 2003 .

[39]  Jonathan R Wolpaw,et al.  Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[40]  N. Hoshimiya,et al.  A multichannel FES system for the restoration of motor functions in high spinal cord injury patients: a respiration-controlled system for multijoint upper extremity , 1989, IEEE Transactions on Biomedical Engineering.