ECoG data analyses to inform closed-loop BCI experiments for speech-based prosthetic applications

Brain Computer Interfaces (BCIs) assist individuals with motor disabilities by enabling them to control prosthetic devices with their neural activity. Performance of closed-loop BCI systems can be improved by using design strategies that leverage structured and task-relevant neural activity. We use data from high density electrocorticography (ECoG) grids implanted in three subjects to study sensory-motor activity during an instructed speech task in which the subjects vocalized three cardinal vowel phonemes. We show how our findings relate to the current understanding of speech physiology and functional organization of human sensory-motor cortex. We investigate the effect of behavioral variations on parameters and performance of the decoding model. Our analyses suggest experimental design strategies that may be critical for speech-based BCI performance.

[1]  Pascal O. Vontobel,et al.  A generalized Blahut-Arimoto algorithm , 2003, IEEE International Symposium on Information Theory, 2003. Proceedings..

[2]  Eric Leuthardt,et al.  Spatiotemporal dynamics of electrocorticographic high gamma activity during overt and covert word repetition , 2011, NeuroImage.

[3]  B. Schölkopf,et al.  Voluntary brain regulation and communication with electrocorticogram signals , 2008, Epilepsy & Behavior.

[4]  G. Schalk,et al.  ECoG factors underlying multimodal control of a brain-computer interface , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[5]  Nicholas P. Szrama,et al.  Using the electrocorticographic speech network to control a brain–computer interface in humans , 2011, Journal of neural engineering.

[6]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[7]  Kristofer E. Bouchard,et al.  Functional Organization of Human Sensorimotor Cortex for Speech Articulation , 2013, Nature.

[8]  Edward F Chang,et al.  Speech map in the human ventral sensory-motor cortex , 2014, Current Opinion in Neurobiology.

[9]  Rajesh P. N. Rao,et al.  Spectral Changes in Cortical Surface Potentials during Motor Movement , 2007, The Journal of Neuroscience.

[10]  Bradley Greger,et al.  Decoding spoken words using local field potentials recorded from the cortical surface , 2010, Journal of neural engineering.

[11]  P. Mitra,et al.  Analysis of dynamic brain imaging data. , 1998, Biophysical journal.

[12]  F. Guenther,et al.  Classification of Intended Phoneme Production from Chronic Intracortical Microelectrode Recordings in Speech-Motor Cortex , 2011, Front. Neurosci..

[13]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[14]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[15]  Rajesh P. N. Rao,et al.  Localization and classification of phonemes using high spatial resolution electrocorticography (ECoG) grids , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[16]  G. Schalk,et al.  Decoding vowels and consonants in spoken and imagined words using electrocorticographic signals in humans , 2011, Journal of neural engineering.

[17]  Kristofer E. Bouchard,et al.  Neural decoding of spoken vowels from human sensory-motor cortex with high-density electrocorticography , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[18]  P. Nunez,et al.  Spatial filtering and neocortical dynamics: estimates of EEG coherence , 1998, IEEE Transactions on Biomedical Engineering.

[19]  J. A. Wilson,et al.  Two-dimensional movement control using electrocorticographic signals in humans , 2008, Journal of neural engineering.

[20]  Gerwin Schalk,et al.  A brain–computer interface using electrocorticographic signals in humans , 2004, Journal of neural engineering.

[21]  Robert D Flint,et al.  Direct classification of all American English phonemes using signals from functional speech motor cortex , 2014, Journal of neural engineering.