Voice of Your Brain: Cognitive Representations of Imagined Speech, Overt Speech, and Speech Perception Based on EEG

Every people has their own voice, likewise, brain signals display distinct neural representations for each individual. Although recent studies have revealed the robustness of speechrelated paradigms for efficient brain-computer interface, the distinction on their cognitive representations with practical usability still remains to be discovered. Herein, we investigate the distinct brain patterns from electroencephalography (EEG) during imagined speech, overt speech, and speech perception in terms of subject variations with its practical use of speaker identification from single channel EEG. We performed classification of nine subjects using deep neural network that captures temporalspectral-spatial features from EEG of imagined speech, overt speech, and speech perception. Furthermore, we demonstrated the underlying neural features of individual subjects while performing imagined speech by comparing the functional connectivity and the EEG envelope features. Our results demonstrate the possibility of subject identification from single channel EEG of imagined speech and overt speech. Also, the comparison of the three speech-related paradigms will provide valuable information for the practical use of speech-related brain signals in the further studies.

[1]  Seong-Whan Lee,et al.  Neural Decoding of Imagined Speech and Visual Imagery as Intuitive Paradigms for BCI Communication , 2020, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[2]  B. V. K. Vijaya Kumar,et al.  Subject identification from electroencephalogram (EEG) signals during imagined speech , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[3]  Stephen M. Gordon,et al.  EEGNet: A Compact Convolutional Neural Network for EEG-based Brain-Computer Interfaces , 2021 .

[4]  Panagiotis Artemiadis,et al.  Inferring imagined speech using EEG signals: a new approach using Riemannian manifold features , 2018, Journal of neural engineering.

[5]  Wei Guo,et al.  Classification of EEG Signals on VEP-Based BCI Systems With Broad Learning , 2021, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[6]  Seong-Whan Lee,et al.  EEG Representations of Spatial and Temporal Features in Imagined Speech and Overt Speech , 2019, ACPR.

[7]  Ying Wah Teh,et al.  Text-Independent Speaker Identification Through Feature Fusion and Deep Neural Network , 2020, IEEE Access.

[8]  Ji-Hoon Jeong,et al.  Towards an EEG-based Intuitive BCI Communication System Using Imagined Speech and Visual Imagery , 2019, 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC).

[9]  John Williamson,et al.  EEG dataset and OpenBMI toolbox for three BCI paradigms: an investigation into BCI illiteracy , 2019, GigaScience.

[10]  Dennis J. McFarland,et al.  Brain–computer interfaces for communication and control , 2002, Clinical Neurophysiology.

[11]  Edward F. Chang,et al.  Speech synthesis from neural decoding of spoken sentences , 2019, Nature.

[12]  Yaser Jararweh,et al.  Soft Computing-Based EEG Classification by Optimal Feature Selection and Neural Networks , 2019, IEEE Transactions on Industrial Informatics.

[13]  Seong-Whan Lee,et al.  A Real-Time Movement Artifact Removal Method for Ambulatory Brain-Computer Interfaces , 2020, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[14]  Bahar Khalighinejad,et al.  Towards reconstructing intelligible speech from the human auditory cortex , 2019, Scientific Reports.

[15]  Luis Alfredo Moctezuma,et al.  Multi-objective optimization for EEG channel selection and accurate intruder detection in an EEG-based subject identification system , 2020, Scientific Reports.

[16]  Thomas L. Griffiths,et al.  Supplementary Information for Natural Speech Reveals the Semantic Maps That Tile Human Cerebral Cortex , 2022 .

[17]  Klaus-Robert Müller,et al.  The Berlin Brain-Computer Interface (BBCI) – towards a new communication channel for online control in gaming applications , 2007, Multimedia Tools and Applications.

[18]  M. D'Zmura,et al.  Envelope responses in single-trial EEG indicate attended speaker in a ‘cocktail party’ , 2014, Journal of Neural Engineering.

[19]  John Williamson,et al.  A High Performance Spelling System based on EEG-EOG Signals With Visual Feedback , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[20]  Thomas Dierks,et al.  BOLD correlates of EEG alpha phase-locking and the fMRI default mode network , 2009, NeuroImage.

[21]  Heung-Il Suk,et al.  Subject and class specific frequency bands selection for multiclass motor imagery classification , 2011, Int. J. Imaging Syst. Technol..

[22]  Peter Hagoort,et al.  Frequency-specific directed interactions in the human brain network for language , 2017 .

[23]  Seong-Whan Lee,et al.  Audio Dequantization for High Fidelity Audio Generation in Flow-based Neural Vocoder , 2020, INTERSPEECH.

[24]  Luis Villaseñor Pineda,et al.  Subjects identification using EEG-recorded imagined speech , 2019, Expert Syst. Appl..

[25]  Paul Sajda,et al.  Compact convolutional neural networks for classification of asynchronous steady-state visual evoked potentials , 2018, Journal of neural engineering.

[26]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[27]  Debadatta Dash,et al.  Spatial and Spectral Fingerprint in the Brain: Speaker Identification from Single Trial MEG Signals , 2019, INTERSPEECH.

[28]  Heung-Il Suk,et al.  A Novel Bayesian Framework for Discriminative Feature Extraction in Brain-Computer Interfaces , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Tanja Schultz,et al.  Biosignal-Based Spoken Communication: A Survey , 2017, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[30]  Thomas Lunner,et al.  Single-channel in-ear-EEG detects the focus of auditory attention to concurrent tone streams and mixed speech , 2016, bioRxiv.

[31]  Seong-Whan Lee,et al.  Error Correction Regression Framework for Enhancing the Decoding Accuracies of Ear-EEG Brain–Computer Interfaces , 2020, IEEE Transactions on Cybernetics.