Classification of Vowels from Imagined Speech with Convolutional Neural Networks

Imagined speech is a relatively new electroencephalography (EEG) neuro-paradigm, which has seen little use in Brain-Computer Interface (BCI) applications. Imagined speech can be used to allow physically impaired patients to communicate and to use smart devices by imagining desired commands and then detecting and executing those commands in a smart device. The goal of this research is to verify previous classification attempts made and then design a new, more efficient neural network that is noticeably less complex (fewer number of layers) that still achieves a comparable classification accuracy. The classifiers are designed to distinguish between EEG signal patterns corresponding to imagined speech of different vowels and words. This research uses a dataset that consists of 15 subjects imagining saying the five main vowels (a, e, i, o, u) and six different words. Two previous studies on imagined speech classifications are verified as those studies used the same dataset used here. The replicated results are compared. The main goal of this study is to take the proposed convolutional neural network (CNN) model from one of the replicated studies and make it much more simpler and less complex, while attempting to retain a similar accuracy. The pre-processing of data is described and a new CNN classifier with three different transfer learning methods is described and used to classify EEG signals. Classification accuracy is used as the performance metric. The new proposed CNN, which uses half as many layers and less complex pre-processing methods, achieved a considerably lower accuracy, but still managed to outperform the initial model proposed by the authors of the dataset by a considerable margin. It is recommended that further studies investigating classifying imagined speech should use more data and more powerful machine learning techniques. Transfer learning proved beneficial and should be used to improve the effectiveness of neural networks.

[1]  Paul Sajda,et al.  Compact convolutional neural networks for classification of asynchronous steady-state visual evoked potentials , 2018, Journal of neural engineering.

[2]  Luis Alfredo Moctezuma,et al.  Towards an API for EEG-Based Imagined Speech classification , 2018 .

[3]  Francisco Sepulveda,et al.  Classifying speech related vs. idle state towards onset detection in brain-computer interfaces overt, inhibited overt, and covert speech sound production vs. idle state , 2014, 2014 IEEE Biomedical Circuits and Systems Conference (BioCAS) Proceedings.

[4]  Damien Coyle,et al.  Classification of imagined spoken Word-Pairs using Convolutional Neural Networks , 2019, GBCIC.

[5]  Frank Rudzicz,et al.  Classifying phonological categories in imagined and articulated speech , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  Makoto Sato,et al.  Single-trial classification of vowel speech imagery using common spatial patterns , 2009, Neural Networks.

[7]  Tiago H. Falk,et al.  Deep learning-based electroencephalography analysis: a systematic review , 2019, Journal of neural engineering.

[8]  Sebastian Ruder,et al.  An overview of gradient descent optimization algorithms , 2016, Vestnik komp'iuternykh i informatsionnykh tekhnologii.

[9]  Aina Puce,et al.  A Review of Issues Related to Data Acquisition and Analysis in EEG/MEG Studies , 2017, Brain sciences.

[10]  Muhammad Ghulam,et al.  Deep Learning for EEG motor imagery classification based on multi-layer CNNs feature fusion , 2019, Future Gener. Comput. Syst..

[11]  Yar Muhammad,et al.  Controlling Electronic Devices with Brain Rhythms/Electrical Activity Using Artificial Neural Network (ANN) , 2019, Bioengineering.

[12]  Guitao Cao,et al.  A random forest model based classification scheme for neonatal amplitude-integrated EEG , 2014, Biomedical engineering online.

[13]  B. V. K. Vijaya Kumar,et al.  Imagined Speech Classification with EEG Signals for Silent Communication: A Preliminary Investigation into Synthetic Telepathy , 2010, 2010 4th International Conference on Bioinformatics and Biomedical Engineering.

[14]  Michael D'Zmura,et al.  EEG-Based Discrimination of Imagined Speech Phonemes , 2011 .

[15]  Athanasios V. Vasilakos,et al.  Brain computer interface: control signals review , 2017, Neurocomputing.

[16]  Iván E. Gareis,et al.  Open access database of EEG signals recorded during imagined speech , 2017, Symposium on Medical Information Processing and Analysis.

[17]  Raffaella Folli,et al.  Optimizing Layers Improves CNN Generalization and Transfer Learning for Imagined Speech Decoding from EEG , 2019, 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC).

[18]  Robert Bogue Brain-computer interfaces: control by thought , 2010, Ind. Robot.

[19]  Damien Coyle,et al.  Mel Frequency Cepstral Coefficients Enhance Imagined Speech Decoding Accuracy from EEG , 2018, 2018 29th Irish Signals and Systems Conference (ISSC).

[20]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[21]  Edward F. Chang,et al.  Speech synthesis from neural decoding of spoken sentences , 2019, Nature.

[22]  Banghua Yang,et al.  Automatic ocular artifacts removal in EEG using deep learning , 2018, Biomed. Signal Process. Control..

[23]  Lingli Yu,et al.  Applying Extreme Learning Machine to classification of EEG BCI , 2016, 2016 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER).

[24]  Christopher C. Cline,et al.  Noninvasive neuroimaging enhances continuous neural tracking for robotic device control , 2019, Science Robotics.