Spontaneous speech dialogue system TOSBURG II and its evaluation

Abstract We have developed a spontaneous speech dialogue system TOSBURG II, employing keyword-based spontaneous speech understanding and multimodal response generation, with adaptive speech response cancellation. Since in multimodal interaction, the user understands the system's response by a visual output before its speech response is completed, the user often interrupts the system's speech response. Therefore, our adaptive speech response cancellation serves to facilitate natural human-computer interaction by allowing the user's interruption. We have also developed an evaluation environment for dialogue data collection and the performance of TOSBURG II. Unlike conventional data collection systems, TOSBURG II collects in this environment not only speech data and the final results of speech understanding but also its intermediate results as dialogue data, to use them for the evaluation and improvement of the system. The results of our dialogue experiments using TOSBURG II prove the effectiveness of adaptive speech response cancellation for natural interaction, confirming that the dialogue data and the evaluation environment will contribute to a further development of spontaneous speech dialogue systems.