Inner speech recognition through electroencephalographic signals

This work focuses on inner speech recognition starting from EEG signals. Inner speech recognition is defined as the internalized process in which the person thinks in pure meanings, generally associated with an auditory imagery of own inner"voice". The decoding of the EEG into text should be understood as the classification of a limited number of words (commands) or the presence of phonemes (units of sound that make up words). Speech-related BCIs provide effective vocal communication strategies for controlling devices through speech commands interpreted from brain signals, improving the quality of life of people who have lost the capability to speak, by restoring communication with their environment. Two public inner speech datasets are analysed. Using this data, some classification models are studied and implemented starting from basic methods such as Support Vector Machines, to ensemble methods such as the eXtreme Gradient Boosting classifier up to the use of neural networks such as Long Short Term Memory (LSTM) and Bidirectional Long Short Term Memory (BiLSTM). With the LSTM and BiLSTM models, generally not used in the literature of inner speech recognition, results in line with or superior to those present in the stateof-the-art are obtained.

[1]  Maryam Alimardani,et al.  Inner Speech Classification using EEG Signals: A Deep Learning Approach , 2021, 2021 IEEE 2nd International Conference on Human-Machine Systems (ICHMS).

[2]  A. G. Ramakrishnan,et al.  Decoding Covert Speech From EEG-A Comprehensive Review , 2021, Frontiers in Neuroscience.

[3]  Rubén D. Spies,et al.  “Thinking out loud”: an open-access EEG-based BCI dataset for inner speech recognition , 2021, bioRxiv.

[4]  Raffaella Folli,et al.  Evaluation of Hyperparameter Optimization in Machine and Deep Learning Methods for Decoding Imagined Speech EEG , 2020, Sensors.

[5]  Chin-Teng Lin,et al.  EEG-Based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and Their Applications , 2020, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[6]  Muhammad Abdul-Mageed,et al.  SPEAK YOUR MIND! Towards Imagined Speech Recognition With Hierarchical Deep Learning , 2019, INTERSPEECH.

[7]  José del R. Millán,et al.  Decoding Inner Speech Using Electrocorticography: Progress and Challenges Toward a Speech Prosthesis , 2018, Front. Neurosci..

[8]  Rupert Ortner,et al.  A New Method to Generate Artificial Frames Using the Empirical Mode Decomposition for an EEG-Based Motor Imagery BCI , 2018, Front. Neurosci..

[9]  Panagiotis Artemiadis,et al.  Inferring imagined speech using EEG signals: a new approach using Riemannian manifold features , 2018, Journal of neural engineering.

[10]  Tanja Schultz,et al.  Biosignal-Based Spoken Communication: A Survey , 2017, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[11]  Luis Villaseñor Pineda,et al.  Sonification and textification: Proposing methods for classifying unspoken words from EEG signals , 2017, Biomed. Signal Process. Control..

[12]  Evelina Fedorenko,et al.  An asymmetrical relationship between verbal and visual thinking: Converging evidence from behavior and fMRI , 2017, NeuroImage.

[13]  Iván E. Gareis,et al.  Open access database of EEG signals recorded during imagined speech , 2017, Symposium on Medical Information Processing and Analysis.

[14]  Brent Lance,et al.  EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces , 2016, Journal of neural engineering.

[15]  Laurent Girin,et al.  Key considerations in designing a speech brain-computer interface , 2016, Journal of Physiology-Paris.

[16]  Snehanshu Saha,et al.  A Communication Paradigm Using Subvocalized Speech: Translating Brain Signals into Speech , 2016, Augmented Human Research.

[17]  Omar Farooq,et al.  Vowel classification using wavelet decomposition during speech imagery , 2016, 2016 3rd International Conference on Signal Processing and Integrated Networks (SPIN).

[18]  Ben Alderson-Day,et al.  Inner Speech: Development, Cognitive Functions, Phenomenology, and Neurobiology , 2015, Psychological bulletin.

[19]  Amir Ali Khan,et al.  Inter comparison of classification techniques for vowel speech imagery using EEG sensors , 2014, The 2014 2nd International Conference on Systems and Informatics (ICSAI 2014).

[20]  Boreom Lee,et al.  EEG classification in a single-trial basis for vowel speech perception using multivariate empirical mode decomposition , 2014, Journal of neural engineering.

[21]  Yu Zhang,et al.  Analysis and classification of speech imagery EEG for BCI , 2013, Biomed. Signal Process. Control..

[22]  M. Kann,et al.  IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB), Special Issue BSB 2012 , 2013 .

[23]  Siyi Deng,et al.  EEG classification of imagined syllable rhythm using Hilbert spectrum methods , 2010, Journal of neural engineering.

[24]  B. V. K. Vijaya Kumar,et al.  Imagined Speech Classification with EEG Signals for Silent Communication: A Preliminary Investigation into Synthetic Telepathy , 2010, 2010 4th International Conference on Bioinformatics and Biomedical Engineering.

[25]  Y. Koike,et al.  Single-trial classification of vowel speech imagery using common spatial patterns , 2009, Neural Networks.

[26]  Michael D'Zmura,et al.  Toward EEG Sensing of Imagined Speech , 2009, HCI.

[27]  Makoto Sato,et al.  Spatial filtering and single-trial classification of EEG during vowel speech imagery , 2009, i-CREATe.

[28]  P Suppes,et al.  Brain wave recognition of words. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[29]  Luis Alfredo Moctezuma,et al.  Towards an API for EEG-Based Imagined Speech classification , 2018 .

[30]  May Salama,et al.  Recognition of Unspoken Words Using Electrode Electroencephalograhic Signals , 2014 .