Classification of voiceless speech using facial muscle activity and vision based techniques

This paper presents a silent speech recognition technique based on facial muscle activity and video, without evaluating any voice signals. This research examines the use of facial surface electromyogram (SEMG) to identify unvoiced vowels and vision-based technique to classify unvoiced consonants. The moving root mean square (RMS) of SEMG signals of four facial muscles is used to segment the signals and to identify the start and end of a silently spoken vowels. Visual features are extracted from the mouth video of a speaker silently uttering consonants using motion segmentation and image moment techniques. The SEMG features and visual features are classified using feedforward multilayer perceptron (MLP) neural networks. The preliminary results demonstrate that the proposed technique yields high recognition rate for classification of unvoiced vowels using SEMG features. Similarly, promising results are obtained in identification of consonants using visual features. The results demonstrate that the system is easy to train for a new user and suggest that such a system works reliably for voiceless, simple speech based commands for human computer interface when it is trained for a user.