Multiclass Classification of Word Imagination Speech With Hybrid Connectivity Features

<italic>Objective:</italic> In this study, electroencephalography data of imagined words were classified using four different feature extraction approaches. Eight subjects were recruited for the recording of imagination with five different words, namely; “go,” “back,” “left,” “right,” and “stop.” <italic>Methods:</italic> One hundred trials for each word were recorded for both imagination and perception, although this study utilized only imagination data. Two different connectivity methods were applied, namely; a covariance-based and a maximum linear cross-correlation-based connectivity measure. These connectivity measures were further computed to extract the phase-only data as an additional method of feature extraction. In addition, four different channel selections were used. The final connectivity matrix from each of the four methods was vectorized and used as the feature vector for the classifier. To classify EEG data, a sigmoid activation function-based linear extreme learning machine was used. <italic>Result and Significance:</italic> We achieved a maximum classification rate of 40.30% (<italic>p</italic> <inline-formula><tex-math notation="LaTeX">$<$ </tex-math></inline-formula> 0.007) and 87.90% (<italic>p</italic> <inline-formula><tex-math notation="LaTeX"> $<$</tex-math></inline-formula> 0.003) in multiclass (five classes) and binary settings, respectively. Thus, our results suggested that EEG responses to imagined speech could be successfully classified using an extreme learning machine. <italic>Conclusion:</italic> This study involving the classification of imagined words can be a milestone contribution toward the development of practical brain–computer interface systems using silent speech.

[1]  G. Hickok Computational neuroanatomy of speech production , 2012, Nature Reviews Neuroscience.

[2]  Hui Wang,et al.  An Adaptive Support Vector Machine-Based Workpiece Surface Classification System Using High-Definition Metrology , 2015, IEEE Transactions on Instrumentation and Measurement.

[3]  Boreom Lee,et al.  EEG classification for motor imagery BCI using phase-only features extracted by independent component analysis , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[4]  C. Meisel,et al.  Scaling Effects and Spatio-Temporal Multilevel Dynamics in Epileptic Seizures , 2012, PloS one.

[5]  Frank H. Guenther,et al.  Brain-computer interfaces for speech communication , 2010, Speech Commun..

[6]  N. Birbaumer,et al.  Brain–computer interfaces for communication and rehabilitation , 2016, Nature Reviews Neurology.

[7]  E. M. Wright,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[8]  G. Pfurtscheller,et al.  Motor imagery and action observation: Modulation of sensorimotor brain rhythms during mental control of a brain–computer interface , 2009, Clinical Neurophysiology.

[9]  Tzyy-Ping Jung,et al.  Independent Component Analysis of Electroencephalographic Data , 1995, NIPS.

[10]  Hang Joon Jo,et al.  Multi-modal, Multi-measure, and Multi-class Discrimination of ADHD with Hierarchical Feature Extraction and Extreme Learning Machine Using Structural and Functional Brain MRI , 2017, Front. Hum. Neurosci..

[11]  G. Schalk,et al.  Decoding vowels and consonants in spoken and imagined words using electrocorticographic signals in humans , 2011, Journal of neural engineering.

[12]  Hang Joon Jo,et al.  ADHD subgroup discrimination with global connectivity features using hierarchical extreme learning machine: Resting-state FMRI study , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[13]  Hang Joon Jo,et al.  Multimodal Discrimination of Schizophrenia Using Hybrid Weighted Feature Concatenation of Brain Functional Connectivity and Anatomical Features with an Extreme Learning Machine , 2017, Front. Neuroinform..

[14]  Wei Cheng,et al.  EEG classification for motor imagery and resting state in BCI applications using multi-class Adaboost extreme learning machine. , 2016, The Review of scientific instruments.

[15]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[16]  Clemens Brunner,et al.  Mu rhythm (de)synchronization and EEG single-trial classification of different motor imagery tasks , 2006, NeuroImage.

[17]  Thea Radüntz,et al.  Automated EEG artifact elimination by applying machine learning algorithms to ICA-based features , 2017, Journal of neural engineering.

[18]  Michael D'Zmura,et al.  Toward EEG Sensing of Imagined Speech , 2009, HCI.

[19]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[20]  B. V. K. Vijaya Kumar,et al.  Imagined Speech Classification with EEG Signals for Silent Communication: A Preliminary Investigation into Synthetic Telepathy , 2010, 2010 4th International Conference on Bioinformatics and Biomedical Engineering.

[21]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[22]  Li Wang,et al.  Improvement of mental tasks with relevant speech imagery for brain-computer interfaces , 2016 .

[23]  Siyi Deng,et al.  EEG classification of imagined syllable rhythm using Hilbert spectrum methods , 2010, Journal of neural engineering.

[24]  Luis Villaseñor Pineda,et al.  Sonification and textification: Proposing methods for classifying unspoken words from EEG signals , 2017, Biomed. Signal Process. Control..

[25]  Cathy J. Price,et al.  A review and synthesis of the first 20 years of PET and fMRI studies of heard speech, spoken language and reading , 2012, NeuroImage.

[26]  Hang Joon Jo,et al.  Multiclass Classification for the Differential Diagnosis on the ADHD Subtypes Using Recursive Feature Elimination and Hierarchical Extreme Learning Machine: Structural MRI Study , 2016, PloS one.

[27]  V. Jousmäki,et al.  Involvement of Primary Motor Cortex in Motor Imagery: A Neuromagnetic Study , 1997, NeuroImage.

[28]  Lifeng Xi,et al.  A selective multiclass support vector machine ensemble classifier for engineering surface classification using high definition metrology , 2015 .

[29]  José del R. Millán,et al.  Word pair classification during imagined speech using direct brain recordings , 2016 .

[30]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces , 2007, Journal of neural engineering.

[31]  Sung Chan Jun,et al.  EEG datasets for motor imagery brain–computer interface , 2017, GigaScience.