Simulation Experiment of BCI Based on Imagined Speech EEG Decoding

Brain Computer Interface (BCI) can help patients of neuromuscular diseases restore parts of the movement and communication abilities that they have lost. Most of BCIs rely on mapping brain activities to device instructions, but limited number of brain activities decides the limited abilities of BCIs. To deal with the problem of limited ablility of BCI, this paper verified the feasibility of constructing BCI based on decoding imagined speech electroencephalography (EEG). As sentences decoded from EEG can have rich meanings, BCIs based on EEG decoding can achieve numerous control instructions. By combining a modified EEG feature extraction mehtod with connectionist temporal classification (CTC), this paper simulated decoding imagined speech EEG using synthetic EEG data without help of speech signal. The performance of decoding model over synthetic data to a certain extent demonstrated the feasibility of constructing BCI based on imagined speech brain signal.

[1]  Brent Lance,et al.  EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces , 2016, Journal of neural engineering.

[2]  Marco Congedo,et al.  Identification of spatial and temporal features of EEG , 2012, Neurocomputing.

[3]  Tanja Schultz,et al.  Brain-to-text: decoding spoken phrases from phone representations in the brain , 2015, Front. Neurosci..

[4]  Yijun Wang,et al.  Common Spatial Pattern Method for Channel Selelction in Motor Imagery Based Brain-computer Interface , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[5]  Jürgen Schmidhuber,et al.  Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks , 2006, ICML.

[6]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[7]  Iñaki Iturrate,et al.  Word pair classification during imagined speech using direct brain recordings , 2016, Scientific Reports.

[8]  F.C. Morabito,et al.  Brain Activity Investigation by EEG Processing: Wavelet Analysis, Kurtosis and Renyi's Entropy for Artifact Detection , 2007, 2007 International Conference on Information Acquisition.

[9]  Makoto Sato,et al.  Single-trial classification of vowel speech imagery using common spatial patterns , 2009, Neural Networks.

[10]  Kemal Polat,et al.  Classification of epileptiform EEG using a hybrid system based on decision tree classifier and fast Fourier transform , 2007, Appl. Math. Comput..

[11]  Erich Elsen,et al.  Deep Speech: Scaling up end-to-end speech recognition , 2014, ArXiv.

[12]  Siyi Deng,et al.  EEG classification of imagined syllable rhythm using Hilbert spectrum methods , 2010, Journal of neural engineering.

[13]  Abdulhamit Subasi,et al.  EEG signal classification using PCA, ICA, LDA and support vector machines , 2010, Expert Syst. Appl..

[14]  G Pfurtscheller,et al.  Separability of EEG signals recorded during right and left motor imagery using adaptive autoregressive parameters. , 1998, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[15]  Wilbert Jan Heeringa.  Measuring dialect pronunciation differences using Levenshtein distance , 2004 .

[16]  Shimon Whiteson,et al.  LipNet: Sentence-level Lipreading , 2016, ArXiv.

[17]  Shimon Whiteson,et al.  LipNet: End-to-End Sentence-level Lipreading , 2016, 1611.01599.