Point-and-Click Cursor Control With an Intracortical Neural Interface System by Humans With Tetraplegia

We present a point-and-click intracortical neural interface system (NIS) that enables humans with tetraplegia to volitionally move a 2-D computer cursor in any desired direction on a computer screen, hold it still, and click on the area of interest. This direct brain-computer interface extracts both discrete (click) and continuous (cursor velocity) signals from a single small population of neurons in human motor cortex. A key component of this system is a multi-state probabilistic decoding algorithm that simultaneously decodes neural spiking activity of a small population of neurons and outputs either a click signal or the velocity of the cursor. The algorithm combines a linear classifier, which determines whether the user is intending to click or move the cursor, with a Kalman filter that translates the neural population activity into cursor velocity. We present a paradigm for training the multi-state decoding algorithm using neural activity observed during imagined actions. Two human participants with tetraplegia (paralysis of the four limbs) performed a closed-loop radial target acquisition task using the point-and-click NIS over multiple sessions. We quantified point-and-click performance using various human-computer interaction measurements for pointing devices. We found that participants could control the cursor motion and click on specified targets with a small error rate (<; 3% in one participant). This study suggests that signals from a small ensemble of motor cortical neurons ( ~ 40) can be used for natural point-and-click 2-D cursor control of a personal computer.

[1]  Dawn M. Taylor,et al.  Direct Cortical Control of 3D Neuroprosthetic Devices , 2002, Science.

[2]  A P Georgopoulos,et al.  On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[3]  A B Schwartz,et al.  Direct cortical representation of drawing. , 1994, Science.

[4]  Michael J. Black,et al.  Decoding Complete Reach and Grasp Actions from Local Primary Motor Cortex Populations , 2010, The Journal of Neuroscience.

[5]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[6]  Apostolos P Georgopoulos,et al.  Erratum to “Directional tuning profiles of motor cortical cells” [Neuroscience Research 36 (2000) 73–79] , 2000, Neuroscience Research.

[7]  R. Andersen,et al.  Cognitive Control Signals for Neural Prosthetics , 2004, Science.

[8]  I. Scott MacKenzie,et al.  Accuracy measures for evaluating computer pointing devices , 2001, CHI.

[9]  Bagrat Amirikian,et al.  Directional tuning profiles of motor cortical cells , 2000, Neuroscience Research.

[10]  Byron M. Yu,et al.  Mixture of Trajectory Models for Neural Decoding of Goal-directed Movements a Computational Model of Craving and Obsession Decoding Visual Inputs from Multiple Neurons in the Human Temporal Lobe Encoding Contribution of Individual Retinal Ganglion Cell Responses to Velocity and Acceleration , 2008 .

[11]  Dean J Krusienski,et al.  Emulation of computer mouse control with a noninvasive brain–computer interface , 2008, Journal of neural engineering.

[12]  Jonathan R Wolpaw,et al.  Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[13]  Prabhat,et al.  Inferring Attentional State and Kinematics from Motor Cortical Firing Rates , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[14]  Dennis C. Tkach,et al.  Congruent Activity during Action and Action Observation in Motor Cortex , 2007, The Journal of Neuroscience.

[15]  A B Schwartz,et al.  Motor cortical representation of speed and direction during reaching. , 1999, Journal of neurophysiology.

[16]  Emery N. Brown,et al.  A State-Space Analysis for Reconstruction of Goal-Directed Movements Using Neural Signals , 2006, Neural Computation.

[17]  J. Donoghue,et al.  Primary Motor Cortex Tuning to Intended Movement Kinematics in Humans with Tetraplegia , 2008, The Journal of Neuroscience.

[18]  Sarah A. Douglas,et al.  Testing pointing device performance and user assessment with the ISO 9241, Part 9 standard , 1999, CHI '99.

[19]  Michael J. Black,et al.  Multi-state decoding of point-and-click control signals from motor cortical activity in a human with tetraplegia , 2007, 2007 3rd International IEEE/EMBS Conference on Neural Engineering.

[20]  R E Kass,et al.  Recursive bayesian decoding of motor cortical signals by particle filtering. , 2004, Journal of neurophysiology.

[21]  Michael J. Black,et al.  Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia , 2008, Journal of neural engineering.

[22]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[23]  Andrew S. Whitford,et al.  Cortical control of a prosthetic arm for self-feeding , 2008, Nature.

[24]  Byron M. Yu,et al.  A high-performance brain–computer interface , 2006, Nature.

[25]  R G Radwin,et al.  Evaluation of a modified Fitts law brain–computer interface target acquisition task in able and motor disabled individuals , 2009, Journal of neural engineering.

[26]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[27]  J. A. Wilson,et al.  Two-dimensional movement control using electrocorticographic signals in humans , 2008, Journal of neural engineering.

[28]  Gerwin Schalk,et al.  A brain–computer interface using electrocorticographic signals in humans , 2004, Journal of neural engineering.

[29]  L. Paninski,et al.  Spatiotemporal tuning of motor cortical neurons for hand position and velocity. , 2004, Journal of neurophysiology.

[30]  David M. Santucci,et al.  Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates , 2003, PLoS biology.

[31]  Wei Wu,et al.  Bayesian Population Decoding of Motor Cortical Activity Using a Kalman Filter , 2006, Neural Computation.

[32]  José Carlos Príncipe,et al.  Bimodal brain-machine interface for motor control of robotic prosthetic , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[33]  S. Meagher Instant neural control of a movement signal , 2002 .

[34]  E. Fetz,et al.  Direct control of paralyzed muscles by cortical neurons , 2008, Nature.

[35]  L.J. Trejo,et al.  Brain-computer interfaces for 1-D and 2-D cursor control: designs using volitional control of the EEG spectrum or steady-state visual evoked potentials , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.