Adapt - a multimodal conversational dialogue system in an apartment domain

A general overview of the AdApt project and the research that is performed within the project is presented. In this project various aspects of human-computer interaction in a multimodal conversational dialogue systems are investigated. The project will also include studies on the integration of user/system/dialogue dependent speech recognition and multimodal speech synthesis. A domain in which multimodal interaction is highly useful has been chosen, namely, finding available apartments in Stockholm. A Wizard-of-Oz data collection within this domain is also described.

[1]  Clifford Nass,et al.  Maximized Modality or constrained consistency? , 1999, AVSP.

[2]  Joakim Gustafson,et al.  Speech technology on trial: Experiences from the August system , 2000, Natural Language Engineering.

[3]  Mark Steedman,et al.  Generating Facial Expressions for Speech , 1996, Cogn. Sci..

[4]  Sangkyu Park,et al.  Multimodal user interfaces in the Open Agent Architecture , 1997, IUI '97.

[5]  Manny Rayner,et al.  Language-Processing Strategies and Mixed-Initiative Dialogues , 1999, Electron. Trans. Artif. Intell..

[6]  Johanna D. Moore,et al.  Multimedia Explanations in IDEA Decision Support Systems , 1998 .

[7]  Sheri Hunnicutt,et al.  An experimental dialogue system: waxholm , 1993, EUROSPEECH.

[8]  Jonas Beskow,et al.  Rule-based visual speech synthesis , 1995, EUROSPEECH.

[9]  Linda Bell,et al.  Modality Convergence in a Multimodal Dialogue System , 2000 .

[10]  Sharon Oviatt,et al.  Integration and synchronization of input modes during multimodal human-computer interaction , 1997 .

[11]  Peter Haddawy,et al.  Interactive and Mixed-Initiative Decision-Theoretic Systems , 1998, AI Mag..

[12]  Joakim Gustafson,et al.  A comparison of disfluency distribution in a unimodal and a multimodal speech interface , 2000, INTERSPEECH.

[13]  Johanna D. Moore,et al.  Working Notes of the AAAI Spring Symposium on Interactive and Mixed-Initiative Decision Theoretic Systems , 1998 .

[14]  Justine Cassell,et al.  Human conversation as a system framework: designing embodied conversational agents , 2001 .

[15]  Joakim Gustafson,et al.  Positive and negative user feedback in a spoken dialogue corpus , 2000, INTERSPEECH.

[16]  Alexander Seward.  A tree-trellis n-best decoder for stochastic context-free grammars , 2000, INTERSPEECH.

[17]  Jonas Beskow,et al.  Animation of talking agents , 1997, AVSP.