Myoelectric signals for multimodal speech recognition

A Coupled Hidden Markov Model (CHMM) is proposed in this paper to perform multimodal speech recognition using myoeletric signals (MES) from the muscles of vocal articulation. MES signals are immune to noise, and words that are acoustically similar manifest distinctly in MES. Hence, they would effectively complement the acoustic data in a multimodal speech recognition system. Research in Audio-Visual Speech Recognition has shown that CHMMs model the asynchrony between different data streams effectively. Hence, we propose CHMM for multimodal speech recognition using audio and MES as the two data streams. Our experiments indicate that the multimodal CHMM system significantly outperforms the audio only system at different SNRs. We have also provided a comparison between different features for MES and have found that wavelet features provide the best results.

[1]  John H. L. Hansen,et al.  HMM-based stressed speech modeling with application to improved synthesis and recognition of isolated speech under stress , 1998, IEEE Trans. Speech Audio Process..

[2]  Thomas S. Huang,et al.  Audio-visual speech modeling using coupled hidden Markov models , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[3]  Minyue Fu,et al.  The use of wavelet transforms in phoneme recognition , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[4]  R.N. Scott,et al.  A new strategy for multifunction myoelectric control , 1993, IEEE Transactions on Biomedical Engineering.

[5]  Kevin P. Murphy,et al.  A coupled HMM for audio-visual speech recognition , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[6]  B. Hudgins,et al.  Hidden Markov model classification of myoelectric signals in speech , 2001, IEEE Engineering in Medicine and Biology Magazine.

[7]  Carlo J. De Luca,et al.  Physiology and Mathematics of Myoelectric Signals , 1979 .

[8]  Yariv Ephraim,et al.  A Bayesian estimation approach for speech enhancement using hidden Markov models , 1992, IEEE Trans. Signal Process..

[9]  B. Hudgins,et al.  Hidden Markov model classification of myoelectric signals in speech , 2002 .

[10]  C. D. De Luca.  Physiology and Mathematics of Myoelectric Signals , 1979, IEEE Transactions on Biomedical Engineering.

[11]  K. Englehart,et al.  Classification of the myoelectric signal using time-frequency based representations. , 1999, Medical engineering & physics.

[12]  Richard Kronland-Martinet,et al.  Analysis of Sound Patterns through Wavelet transforms , 1987, Int. J. Pattern Recognit. Artif. Intell..