Using phase to recognize English phonemes and their distinctive features in the brain

The neural mechanisms used by the human brain to identify phonemes remain unclear. We recorded the EEG signals evoked by repeated presentation of 12 American English phonemes. A support vector machine model correctly recognized a high percentage of the EEG brain wave recordings represented by their phases, which were expressed in discrete Fourier transform coefficients. We show that phases of the oscillations restricted to the frequency range of 2–9 Hz can be used to successfully recognize brain processing of these phonemes. The recognition rates can be further improved using the scalp tangential electric field and the surface Laplacian around the auditory cortical area, which were derived from the original potential signal. The best rate for the eight initial consonants was 66.7%. Moreover, we found a distinctive phase pattern in the brain for each of these consonants. We then used these phase patterns to recognize the consonants, with a correct rate of 48.7%. In addition, in the analysis of the confusion matrices, we found significant similarity–differences were invariant between brain and perceptual representations of phonemes. These latter results supported the importance of phonological distinctive features in the neural representation of phonemes.

[1]  Patrick Suppes,et al.  Phase-oscillator computations as neural models of stimulus–response conditioning and response selection ☆ , 2010, 1010.3063.

[2]  Jont B. Allen,et al.  Consonant confusions in white noise. , 2008, The Journal of the Acoustical Society of America.

[3]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[4]  Patrick Suppes,et al.  Partial Orders of Similarity Differences Invariant Between EEG-Recorded Brain and Perceptual Representations of Language , 2009, Neural Computation.

[5]  Jonas Obleser,et al.  Magnetic Brain Response Mirrors Extraction of Phonological Features from Spoken Vowels , 2004, Journal of Cognitive Neuroscience.

[6]  R. Ilmoniemi,et al.  Language-specific phoneme representations revealed by electric and magnetic brain responses , 1997, Nature.

[7]  Eric Halgren,et al.  Linear Coding of Voice Onset Time , 2007, Journal of Cognitive Neuroscience.

[8]  Rainer Goebel,et al.  "Who" Is Saying "What"? Brain-Based Decoding of Human Voice and Speech , 2008, Science.

[9]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[10]  M. D. Wang,et al.  Consonant confusions in noise: a study of perceptual features. , 1973, The Journal of the Acoustical Society of America.

[11]  R. Shiffrin,et al.  A retrieval model for both recognition and recall. , 1984, Psychological review.

[12]  C. Schroeder,et al.  Tonotopic organization of responses reflecting stop consonant place of articulation in primary auditory cortex (A1) of the monkey , 1995, Brain Research.

[13]  P. Hagoort,et al.  Oscillatory neuronal dynamics during language comprehension. , 2006, Progress in brain research.

[14]  G. A. Miller,et al.  An Analysis of Perceptual Confusions Among Some English Consonants , 1955 .

[15]  L. Lisker,et al.  A Cross-Language Study of Voicing in Initial Stops: Acoustical Measurements , 1964 .

[16]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[17]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[18]  R. Näätänen,et al.  Early selective-attention effect on evoked potential reinterpreted. , 1978, Acta psychologica.

[19]  M. Kilgard,et al.  Cortical activity patterns predict speech discrimination ability , 2008, Nature Neuroscience.

[20]  •'AYNE A. •'ICKELGREN.  Distinctive Features and Errors in Short-Term Memory for English Consonants , 2004 .

[21]  Patrick Suppes,et al.  A Spline Framework for Estimating the EEG Surface Laplacian Using the Euclidean Metric , 2011, Neural Computation.

[22]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[23]  D. Poeppel,et al.  Phase Patterns of Neuronal Responses Reliably Discriminate Speech in Human Auditory Cortex , 2007, Neuron.

[24]  T. Sejnowski,et al.  Removing electroencephalographic artifacts by blind source separation. , 2000, Psychophysiology.

[25]  J. Fell,et al.  Rhinal–hippocampal theta coherence during declarative memory formation: interaction with gamma synchronization? , 2003, The European journal of neuroscience.

[26]  Alex A. Freitas,et al.  A survey of hierarchical classification across different application domains , 2010, Data Mining and Knowledge Discovery.

[27]  Marc W Howard,et al.  Theta and Gamma Oscillations during Encoding Predict Subsequent Recall , 2003, The Journal of Neuroscience.

[28]  J. Arezzo,et al.  Representation of the voice onset time (VOT) speech parameter in population responses within primary auditory cortex of the awake monkey. , 2003, The Journal of the Acoustical Society of America.

[29]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[30]  Carsten Eulitz.  REPRESENTATION OF PHONOLOGICAL FEATURES IN THE BRAIN: EVIDENCE FROM MISMATCH NEGATIVITY , 2007 .

[31]  Linguistic lapses: With especial reference to the perception of linguistic sounds , 1906 .

[32]  J. Fell,et al.  The role of phase synchronization in memory processes , 2011, Nature Reviews Neuroscience.

[33]  David A. Medler,et al.  Cerebral Cortex doi:10.1093/cercor/bhi040 Cerebral Cortex Advance Access published February 9, 2005 , 2022 .

[34]  Nima Mesgarani,et al.  Phoneme representation and classification in primary auditory cortex. , 2008, The Journal of the Acoustical Society of America.

[35]  R. Shiffrin,et al.  A model for recognition memory: REM—retrieving effectively from memory , 1997, Psychonomic bulletin & review.

[36]  James M Kilner,et al.  Event-related brain dynamics , 2002, Trends in Neurosciences.

[37]  O. Bertrand,et al.  Oscillatory gamma activity in humans and its role in object representation , 1999, Trends in Cognitive Sciences.

[38]  Coarticulation • Suprasegmentals,et al.  Acoustic Phonetics , 2019, The SAGE Encyclopedia of Human Communication Sciences and Disorders.

[39]  R. Näätänen.  The perception of speech sounds by the human brain as reflected by the mismatch negativity (MMN) and its magnetic equivalent (MMNm). , 2001, Psychophysiology.

[40]  J. M. Pickett,et al.  Perception of Vowels Heard in Noises of Various Spectra , 1957 .

[41]  R. Oostenveld,et al.  Theta and Gamma Oscillations Predict Encoding and Retrieval of Declarative Memory , 2006, The Journal of Neuroscience.

[42]  W. Klimesch,et al.  What does phase information of oscillatory brain activity tell us about cognitive processes? , 2008, Neuroscience & Biobehavioral Reviews.

[43]  D. Tucker.  Spatial sampling of head electrical fields: the geodesic sensor net. , 1993, Electroencephalography and clinical neurophysiology.

[44]  Patrick Suppes,et al.  Learning Pattern Recognition Through Quasi-Synchronization of Phase Oscillators , 2011, IEEE Transactions on Neural Networks.