Cortical encoding of phonemic context during word production

Brain-computer interfaces that directly decode speech could restore communication to locked-in individuals. However, decoding speech from brain signals still faces many challenges. We investigated decoding of phonemes - the smallest separable parts of speech - from ECoG signals during word production. We expanded on previous efforts to identify specific phoneme by identifying phonemes by where in the word they were formed. We evaluated how the context of phonemes in words affects classification results using linear discriminant analysis. The decoding accuracy of our linear classifier indicated the degree to which the context of a phoneme can be determined from the cortical signal significantly greater than chance. Further, we identified the spectrotemporal features that contributed most to successful decoding of phonemic classes. Finally, we discuss how this can augment speech decoding for neural interfaces.

[1]  Frank H. Guenther,et al.  Brain-computer interfaces for speech communication , 2010, Speech Commun..

[2]  G. Schalk,et al.  Decoding vowels and consonants in spoken and imagined words using electrocorticographic signals in humans , 2011, Journal of neural engineering.

[3]  G. Hickok Computational neuroanatomy of speech production , 2012, Nature Reviews Neuroscience.

[4]  Kristofer E. Bouchard,et al.  Functional Organization of Human Sensorimotor Cortex for Speech Articulation , 2013, Nature.

[5]  W. Penfield,et al.  SOMATIC MOTOR AND SENSORY REPRESENTATION IN THE CEREBRAL CORTEX OF MAN AS STUDIED BY ELECTRICAL STIMULATION , 1937 .

[6]  Adam Brown International Phonetic Alphabet , 2012 .

[7]  Robert D Flint,et al.  Direct classification of all American English phonemes using signals from functional speech motor cortex , 2014, Journal of neural engineering.

[8]  F. Guenther Cortical interactions underlying the production of speech sounds. , 2006, Journal of communication disorders.

[9]  Nick F. Ramsey,et al.  Automated electrocorticographic electrode localization on individually rendered brain surfaces , 2010, Journal of Neuroscience Methods.

[10]  Nicholas P. Szrama,et al.  Using the electrocorticographic speech network to control a brain–computer interface in humans , 2011, Journal of neural engineering.

[11]  Xuan Wei,et al.  Validation of chronic implantable neural sensing technology using electrocorticographic (ECoG) based brain machine interfaces , 2011, 2011 5th International IEEE/EMBS Conference on Neural Engineering.

[12]  Bradley Greger,et al.  Decoding spoken words using local field potentials recorded from the cortical surface , 2010, Journal of neural engineering.

[13]  L. Miller,et al.  Accurate decoding of reaching movements from field potentials in the absence of spikes , 2012, Journal of neural engineering.