MuDiS – A Multimodal Dialogue System for Human–Robot Interaction

We present the MuDiS project. The main goal of MuDiS is to develop a Multimodal Dialogue System that can be adapted quickly to a wide range of various scenarios. In this interdisciplinary project, we unite researchers from diverse areas, including computational linguistics, computer science, electrical engineering, and psychology. The different research lines of MuDiS reflect the interdisciplinary character of the project. In this publication, we show how MuDiS collects data from human-human experiments to get new insights in multimodal human interaction. Furthermore, we present the first version of the MuDiS system architecture, which contains new components for classification of head movements, multimodal fusion, and dialogue management. Finally, we describe the application of the MuDiS system in a human-robot interaction scenario to prove that MuDiS can be implemented in different domains.

[1]  George H. Mealy,et al.  A method for synthesizing sequential circuits , 1955 .

[2]  Jörgen Ahlberg,et al.  CANDIDE-3 - An Updated Parameterised Face , 2001 .

[3]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[4]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[5]  Timothy F. Cootes,et al.  Active Shape Models - 'smart snakes' , 1992, BMVC.

[6]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[7]  Alois Knoll,et al.  Integrating Language, Vision and Action for Human Robot Dialog Systems , 2007, HCI.

[8]  Marilyn A. Walker,et al.  MATCH: An Architecture for Multimodal Dialogue Systems , 2002, ACL.

[9]  Laurent Romary,et al.  Multimodal Meaning Representation for Generic Dialogue Systems Architectures , 2004, LREC.

[10]  Jean-Claude Martin,et al.  Collection and Annotation of a Corpus of Human-Human Multimodal Interactions: Emotion and Others Anthropomorphic Characteristics , 2007, ACII.

[11]  Philip R. Cohen,et al.  QuickSet: multimodal interaction for distributed applications , 1997, MULTIMEDIA '97.

[12]  Louis Vuurpijl,et al.  Multimodal Interaction in Architectural Design Applications , 2004, User Interfaces for All.

[13]  Wolfgang Wahlster,et al.  SmartKom: Foundations of Multimodal Dialogue Systems , 2006, SmartKom.

[14]  Bernd Radig,et al.  Learning Robust Objective Functions with Application to Face Model Fitting , 2007, DAGM-Symposium.

[15]  Kerstin Dautenhahn,et al.  A quantitative technique for analysing robot-human interactions , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Meng-Chang Lee.  Top 100 Documents Browse Search Ieee Xplore Guide Support Top 100 Documents Accessed: Nov 2005 a Tutorial on Hidden Markov Models and Selected Applications Inspeech Recognition , 2005 .

[17]  Richard A. Bolt,et al.  “Put-that-there”: Voice and gesture at the graphics interface , 1980, SIGGRAPH '80.

[18]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[19]  Michael Beetz,et al.  The Contracting Curve Density Algorithm: Fitting Parametric Curve Models to Images Using Local Self-Adapting Separation Criteria , 2004, International Journal of Computer Vision.