Joint Spatial-Spectral Feature Space Clustering for Speech Activity Detection from ECoG Signals

Brain-machine interfaces for speech restoration have been extensively studied for more than two decades. The success of such a system will depend in part on selecting the best brain recording sites and signal features corresponding to speech production. The purpose of this study was to detect speech activity automatically from electrocorticographic signals based on joint spatial-frequency clustering of the ECoG feature space. For this study, the ECoG signals were recorded while a subject performed two different syllable repetition tasks. We found that the optimal frequency resolution to detect speech activity from ECoG signals was 8 Hz, achieving 98.8% accuracy by employing support vector machines as a classifier. We also defined the cortical areas that held the most information about the discrimination of speech and nonspeech time intervals. Additionally, the results shed light on the distinct cortical areas associated with the two syllables repetition tasks and may contribute to the development of portable ECoG-based communication.

[1]  F. Guenther,et al.  A Wireless Brain-Machine Interface for Real-Time Speech Synthesis , 2009, PloS one.

[2]  V. A. Konyshev,et al.  A P300-based brain—computer interface , 2007, Meditsinskaia tekhnika.

[3]  Igor Kononenko,et al.  Estimating Attributes: Analysis and Extensions of RELIEF , 1994, ECML.

[4]  Anil K. Jain,et al.  Artificial Neural Networks: A Tutorial , 1996, Computer.

[5]  Nicholas P. Szrama,et al.  Using the electrocorticographic speech network to control a brain–computer interface in humans , 2011, Journal of neural engineering.

[6]  X. Zeng,et al.  Geometric strategies for neuroanatomic analysis from MRI , 2004, NeuroImage.

[7]  E. Donchin,et al.  A P300-based brain–computer interface: Initial tests by ALS patients , 2006, Clinical Neurophysiology.

[8]  S. Acharya,et al.  Toward Electrocorticographic Control of a Dexterous Upper Limb Prosthesis: Building Brain-Machine Interfaces , 2012, IEEE Pulse.

[9]  E Donchin,et al.  The mental prosthesis: assessing the speed of a P300-based brain-computer interface. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[10]  Ciprian M. Crainiceanu,et al.  Dynamics of large-scale cortical interactions at high gamma frequencies during word production: Event related causality (ERC) analysis of human electrocorticography (ECoG) , 2011, NeuroImage.

[11]  Thomas Stieglitz,et al.  Towards Electrocorticographic Electrodes for Chronic Use in BCI Applications , 2012 .

[12]  B. Gordon,et al.  Induced electrocorticographic gamma activity during auditory perception , 2001, Clinical Neurophysiology.

[13]  Michael H Kohrman,et al.  ECoG gamma activity during a language task: differentiating expressive and receptive speech areas. , 2008, Brain : a journal of neurology.

[14]  N. Birbaumer,et al.  The thought-translation device (TTD): neurobehavioral mechanisms and clinical outcome , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[15]  Afsheen Afshar,et al.  Free-paced high-performance brain-computer interfaces. , 2007, Journal of neural engineering.

[16]  Gustavo P. Sudre,et al.  Decoding semantic information from human electrocorticographic (ECoG) signals , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[17]  V. Gilja,et al.  Signal Processing Challenges for Neural Prostheses , 2008, IEEE Signal Processing Magazine.

[18]  Naotaka Fujii,et al.  Long-Term Asynchronous Decoding of Arm Motion Using Electrocorticographic Signals in Monkeys , 2009, Front. Neuroeng..

[19]  David M. Santucci,et al.  Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates , 2003, PLoS biology.

[20]  S. Acharya,et al.  Connectivity Analysis as a Novel Approach to Motor Decoding for Prosthesis Control , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[21]  Xiaorong Gao,et al.  Design and implementation of a brain-computer interface with high transfer rates , 2002, IEEE Transactions on Biomedical Engineering.

[22]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[23]  Tzyy-Ping Jung,et al.  Biosensor Technologies for Augmented Brain–Computer Interfaces in the Next Decades , 2012, Proceedings of the IEEE.

[24]  J. Wolpaw,et al.  A P300-based brain–computer interface for people with amyotrophic lateral sclerosis , 2008, Clinical Neurophysiology.

[25]  N. Barbaro,et al.  Spatiotemporal Dynamics of Word Processing in the Human Brain , 2007, Front. Neurosci..

[26]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[27]  Bradley G. Goodyear,et al.  Cortical reorganization and reduced efficiency of visual word recognition in right temporal lobe epilepsy: A functional MRI study , 2011, Epilepsy Research.

[28]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[29]  J. Lisman,et al.  The Theta-Gamma Neural Code , 2013, Neuron.

[30]  E. Niebur,et al.  Neural Correlates of High-Gamma Oscillations (60–200 Hz) in Macaque Local Field Potentials and Their Potential Implications in Electrocorticography , 2008, The Journal of Neuroscience.

[31]  G. Schalk,et al.  Decoding vowels and consonants in spoken and imagined words using electrocorticographic signals in humans , 2011, Journal of neural engineering.

[32]  Nick F. Ramsey,et al.  Human Motor Cortical Activity Is Selectively Phase-Entrained on Underlying Rhythms , 2012, PLoS Comput. Biol..

[33]  Rabab K Ward,et al.  A survey of signal processing algorithms in brain–computer interfaces based on electrical brain signals , 2007, Journal of neural engineering.

[34]  Robert T. Knight,et al.  Spatiotemporal imaging of cortical activation during verb generation and picture naming , 2010, NeuroImage.

[35]  D GOLDMAN,et al.  The clinical use of the "average" reference electrode in monopolar recording. , 1950, Electroencephalography and clinical neurophysiology.

[36]  J. Lisman,et al.  The θ-γ neural code. , 2013, Neuron.

[37]  H. Flor,et al.  A spelling device for the paralysed , 1999, Nature.

[38]  M. Stavrinou,et al.  Evaluation of Cortical Connectivity During Real and Imagined Rhythmic Finger Tapping , 2007, Brain Topography.

[39]  Christa Neuper,et al.  Motor imagery and EEG-based control of spelling devices and neuroprostheses. , 2006, Progress in brain research.

[40]  Dejan Markovic,et al.  Spike Sorting: The First Step in Decoding the Brain: The first step in decoding the brain , 2012, IEEE Signal Processing Magazine.

[41]  A. Graser,et al.  Spelling with Steady-State Visual Evoked Potentials , 2007, 2007 3rd International IEEE/EMBS Conference on Neural Engineering.

[42]  J. Seifer,et al.  [Locked-in syndrome]. , 1982, Medicina.

[43]  G. Schalk,et al.  Silent Communication: Toward Using Brain Signals , 2012, IEEE Pulse.

[44]  H. Flor,et al.  The thought translation device (TTD) for completely paralyzed patients. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[45]  J. Wolpaw,et al.  Decoding two-dimensional movement trajectories using electrocorticographic signals in humans , 2007, Journal of neural engineering.

[46]  Eibe Frank,et al.  Logistic Model Trees , 2003, ECML.

[47]  Paul Boersma,et al.  Praat, a system for doing phonetics by computer , 2002 .

[48]  J. Wolpaw,et al.  Decoding flexion of individual fingers using electrocorticographic signals in humans , 2009, Journal of neural engineering.

[49]  X. Papademetris,et al.  Dissociation between the Activity of the Right Middle Frontal Gyrus and the Middle Temporal Gyrus in Processing Semantic Priming , 2011, PloS one.

[50]  Makoto Sato,et al.  Single-trial classification of vowel speech imagery using common spatial patterns , 2009, Neural Networks.

[51]  Christa Neuper,et al.  An asynchronously controlled EEG-based virtual keyboard: improvement of the spelling rate , 2004, IEEE Transactions on Biomedical Engineering.

[52]  R. Bracewell.  The Fourier transform. , 1989, Scientific American.

[53]  Brian N. Pasley,et al.  Reconstructing Speech from Human Auditory Cortex , 2012, PLoS biology.

[54]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces , 2007, Journal of neural engineering.

[55]  G Pfurtscheller,et al.  Using time-dependent neural networks for EEG classification. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[56]  E. W. Sellers,et al.  Toward enhanced P300 speller performance , 2008, Journal of Neuroscience Methods.

[57]  Bradley Greger,et al.  Decoding spoken words using local field potentials recorded from the cortical surface , 2010, Journal of neural engineering.

[58]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[59]  D.J. McFarland,et al.  The wadsworth BCI research and development program: at home with BCI , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[60]  G. Schalk,et al.  Brain-Computer Interfaces Using Electrocorticographic Signals , 2011, IEEE Reviews in Biomedical Engineering.

[61]  N. Birbaumer,et al.  A brain–computer interface (BCI) for the locked-in: comparison of different EEG classifications for the thought translation device , 2003, Clinical Neurophysiology.

[62]  Tzyy-Ping Jung,et al.  Real-World Neuroimaging Technologies , 2013, IEEE Access.

[63]  R. Irizarry,et al.  Electrocorticographic gamma activity during word production in spoken and sign language , 2001, Neurology.