EEG based direct speech BCI system using a fusion of SMRT and MFCC/LPCC features with ANN classifier

Abstract Brain computer interface (BCI) technology has a great deal of scientific interest with various application systems. An advancement that is increasingly relevant in the BCI is communicating with a speech in mind. Thus, the paper aims to develop a direct speech BCI (DS-BCI) system using short time-based features and an ANN classifier. Signal processing methods like Mel frequency cepstral coefficients (MFCC), Linear predictive cepstral coefficients (LPCC) and Sequency mapped real transform (SMRT) are utilized on a short time bases to extract base level features. Statistical parameters are then determined based on the ensemble average (EA) and time average (TA) to extract two reduced vectors in each method. Hybrid feature vectors like MLC, SMC and SLC are prepared by fusion of features from MFCC & LPCC, SMRT & MFCC and SMRT & LPCC, respectively, in both EA & TA analysis. Principal component analysis (PCA) is performed on hybrid feature vectors to derive uncorrelated components. The proposed method is evaluated on imagined EEG (EEG-i) & vocalized EEG (EEG-v) signals from the ‘Kara one’ database and presented classification accuracy of individual methods & hybrid methods. The results show that hybrid features SMC & SLC enhance the classification accuracy compared to the unique features. PCA analysis helps improve accuracy and reduce feature dimension. TA-based SMC features with PCA provide maximum accuracy as 77.37% and 62.52% for EEG-i and EEG-v signals, respectively. The proposed method outperforms the state-of-the-art algorithms discussed in the paper.

[1]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[2]  B. V. K. Vijaya Kumar,et al.  Subject identification from electroencephalogram (EEG) signals during imagined speech , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[3]  H. Hotelling.  Analysis of a complex of statistical variables into principal components. , 1933 .

[4]  Damien Coyle,et al.  Neurolinguistics Research Advancing Development of a Direct-Speech Brain-Computer Interface , 2018, iScience.

[5]  Jaime Gómez Gil,et al.  Brain Computer Interfaces, a Review , 2012, Sensors.

[6]  Yusuf Uzzaman Khan,et al.  EEG based classification of imagined vowel sounds , 2015, 2015 2nd International Conference on Computing for Sustainable Global Development (INDIACom).

[7]  Jongin Kim,et al.  Vowel Imagery Decoding toward Silent Speech BCI Using Extreme Learning Machine with Electroencephalogram , 2016, BioMed research international.

[8]  Luis Villaseñor Pineda,et al.  Sonification and textification: Proposing methods for classifying unspoken words from EEG signals , 2017, Biomed. Signal Process. Control..

[9]  Jonathan S Brumberg,et al.  Brain-Computer Interfaces for Augmentative and Alternative Communication: A Tutorial. , 2018, American journal of speech-language pathology.

[10]  Tanja Schultz,et al.  Automatic Speech Recognition from Neural Signals: A Focused Review , 2016, Front. Neurosci..

[11]  Michael D'Zmura,et al.  Toward EEG Sensing of Imagined Speech , 2009, HCI.

[12]  Frank Rudzicz,et al.  Classifying phonological categories in imagined and articulated speech , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Omar Farooq,et al.  Vowel classification using wavelet decomposition during speech imagery , 2016, 2016 3rd International Conference on Signal Processing and Integrated Networks (SPIN).

[14]  R. Gopikakumari,et al.  A new transform for 2-D signal representation (MRT) and some of its properties , 2004, 2004 International Conference on Signal Processing and Communications, 2004. SPCOM '04..

[15]  J. M. Gilbert,et al.  Silent speech interfaces , 2010, Speech Commun..

[16]  Tanja Schultz,et al.  Biosignal-Based Spoken Communication: A Survey , 2017, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[17]  Boreom Lee,et al.  EEG classification in a single-trial basis for vowel speech perception using multivariate empirical mode decomposition , 2014, Journal of neural engineering.

[18]  Damien Coyle,et al.  Mel Frequency Cepstral Coefficients Enhance Imagined Speech Decoding Accuracy from EEG , 2018, 2018 29th Irish Signals and Systems Conference (ISSC).

[19]  David M. Skapura,et al.  Neural networks - algorithms, applications, and programming techniques , 1991, Computation and neural systems series.

[20]  Boreom Lee,et al.  Multiclass Classification of Word Imagination Speech With Hybrid Connectivity Features , 2018, IEEE Transactions on Biomedical Engineering.

[21]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[22]  John P. John,et al.  Assessing Neurocognition via Gamified Experimental Logic: A Novel Approach to Simultaneous Acquisition of Multiple ERPs , 2016, Front. Neurosci..

[23]  R. Gopikakumari,et al.  Wavelet feature selection of audio and imagined/vocalized EEG signals for ANN based multimodal ASR system , 2021, Biomed. Signal Process. Control..

[24]  Panagiotis Artemiadis,et al.  Inferring imagined speech using EEG signals: a new approach using Riemannian manifold features , 2018, Journal of neural engineering.

[25]  Luis Villaseñor Pineda,et al.  Transfer learning in imagined speech EEG-based BCIs , 2019, Biomed. Signal Process. Control..

[26]  Tessamma Thomas,et al.  Feature Vector Selection of Fusion of MFCC and SMRT Coefficients for SVM Classifier Based Speech Recognition System , 2018, 2018 8th International Symposium on Embedded Computing and System Design (ISED).

[27]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[28]  Sajad Shirali-Shahreza,et al.  Effect of MFCC normalization on vector quantization based speaker identification , 2010, The 10th IEEE International Symposium on Signal Processing and Information Technology.

[29]  Muhammad Abdul-Mageed,et al.  SPEAK YOUR MIND! Towards Imagined Speech Recognition With Hierarchical Deep Learning , 2019, INTERSPEECH.

[30]  Makoto Sato,et al.  Single-trial classification of vowel speech imagery using common spatial patterns , 2009, Neural Networks.

[31]  M. A. Anusuya,et al.  Speech Recognition by Machine, A Review , 2010, ArXiv.

[32]  Chang-Hwan Im,et al.  EEG-Based Brain-Computer Interfaces: A Thorough Literature Survey , 2013, Int. J. Hum. Comput. Interact..

[33]  Iván E. Gareis,et al.  Open access database of EEG signals recorded during imagined speech , 2017, Symposium on Medical Information Processing and Analysis.

[34]  E. Wong,et al.  Comparison of linear prediction cepstrum coefficients and mel-frequency cepstrum coefficients for language identification , 2001, Proceedings of 2001 International Symposium on Intelligent Multimedia, Video and Speech Processing. ISIMP 2001 (IEEE Cat. No.01EX489).

[35]  Tom Chau,et al.  EEG Classification of Covert Speech Using Regularized Neural Networks , 2017, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[36]  Sumeet Agarwal,et al.  Classification and analysis of EEG signals for imagined motor movements , 2015, 2015 IEEE Workshop on Computational Intelligence: Theories, Applications and Future Directions (WCI).

[37]  Jun Qin,et al.  Neural networks based EEG-Speech Models , 2016, ArXiv.

[38]  Jasmin Kevric,et al.  Biomedical Signal Processing and Control , 2016 .

[39]  Carlos A. Reyes-García,et al.  Implementing a fuzzy inference system in a multi-objective EEG channel selection model for imagined speech classification , 2016 .

[40]  Jinchang Ren,et al.  EEG-Based Brain-Computer Interfaces Using Motor-Imagery: Techniques and Challenges , 2019, Sensors.

[41]  Snehanshu Saha,et al.  A Communication Paradigm Using Subvocalized Speech: Translating Brain Signals into Speech , 2016, Augmented Human Research.

[42]  Siyi Deng,et al.  EEG classification of imagined syllable rhythm using Hilbert spectrum methods , 2010, Journal of neural engineering.

[43]  D.R. Reddy,et al.  Speech recognition by machine: A review , 1976, Proceedings of the IEEE.

[44]  Yu Zhang,et al.  Analysis and classification of speech imagery EEG for BCI , 2013, Biomed. Signal Process. Control..

[45]  Philip D. Wasserman,et al.  Neural computing - theory and practice , 1989 .

[46]  K. Meenakshy,et al.  Prostate Disease Diagnosis from CT Images Using GA Optimized SMRT Based Texture Features , 2015 .

[47]  V. A. Makarov,et al.  Recovering EEG brain signals: Artifact suppression with wavelet enhanced independent component analysis , 2006, Journal of Neuroscience Methods.