A system for facial expression-based affective speech translation
暂无分享,去创建一个
In the emerging field of speech-to-speech translation, emphasis is currently placed on the linguistic content, while the significance of paralinguistic information conveyed by facial expression or tone of voice is typically neglected. We present a prototype system for multimodal speech-to-speech translation that is able to automatically recognize and translate spoken utterances from one language into another, with the output rendered by a speech synthesis system. The novelty of our system lies in the technique of generating the synthetic speech output in one of several expressive styles that is automatically determined using a camera to analyze the user's facial expression during speech.
[1] Marc Schröder,et al. The German Text-to-Speech Synthesis System MARY: A Tool for Research, Development and Teaching , 2003, Int. J. Speech Technol..
[2] Julie Carson-Berndsen,et al. Facial expression as an input annotation modality for affective speech-to-speech translation , 2012 .
[3] Christian Küblbeck,et al. Face detection and tracking in video sequences using the modifiedcensus transformation , 2006, Image Vis. Comput..