Client and Speech Detection System for Intelligent Infokiosk

Timely attraction of a client and detection of his/her speech message in real noisy conditions are main difficulties at deployment of speech and multimodal interfaces in information kiosks. Combination of sound source localization, voice activity and face detection technologies allowed to determine client mouth coordinates and extract boundaries of speech signal appeared in the kiosk dialogue area. Talking head model based on audio-visual speech synthesis immediately greets the client, when her face is captured in the videomonitoring area, in order to attract him/her to the information service before leaving the interaction area. Client's face tracking is also used for turning the talking head in direction to the client that significantly improves the naturalness of interaction. The developed infokiosk set in the institute hall provides information about structure and staff of laboratories. Statistics of human-kiosk interaction is accumulated within last six months in 2009.

[1]  Michael Johnston,et al.  MATCHkiosk: A Multimodal Interactive City Guide , 2004, ACL.

[2]  Andrey Ronzhin,et al.  Audio-visual speech asynchrony modeling in a talking head , 2009, INTERSPEECH.

[3]  A. A. Karpov,et al.  Information enquiry kiosk with multimodal user interface , 2009, Pattern Recognition and Image Analysis.

[4]  Jean-Marc Odobez,et al.  Multimodal multispeaker probabilistic tracking in meetings , 2005, ICMI '05.

[5]  Paul A. Viola,et al.  Boosting-Based Multimodal Speaker Detection for Distributed Meeting Videos , 2008, IEEE Transactions on Multimedia.

[6]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[7]  Lee McCauley,et al.  MIKI: A Speech Enabled Intelligent Kiosk , 2006, IVA.

[8]  G. Carter,et al.  The generalized correlation method for estimation of time delay , 1976 .

[9]  Maurizio Omologo,et al.  Acoustic event localization using a crosspower-spectrum phase based technique , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[10]  Iain McCowan,et al.  A sector-based approach for localization of multiple speakers with microphone arrays , 2004, SAPA@INTERSPEECH.