The WITAS multi-modal dialogue system I

We present the first demonstration version of the WITAS dialogue system for multi-modal conversations with autonomous mobile robots, and motivate several innovations currently in development for version II. The human-robot interaction setting is argued to present new challenges for dialogue system engineers, in comparison to previous work in dialogue systems under the travel-planning paradigm, in that dialogues must be asynchronous, mixed-initiative, open-ended, and involve a dynamic environment. We approached these general problems in a dialogue interface to the WITAS robot helicopter, or UAV (‘Unmanned Aerial Vehicle’). We present this system and the modelling ideas behind it, and then motivate changes being made for version II of the system, involving more richly structured dialogue states and the use of automated reasoning systems over task, ability, and world-state models. We argue that these sorts of enhancement are vital to the future development of conversational systems. 1. Dialogues with mobile robots Many dialogue systems have been built for use in contexts where conversational interactions are predictable and can be scripted, and where the operating environment is static. For example, a dialogue for buying an airline flight can be specified by way of filling in certain parameters (cost, destination, and so on) and a database query, report, and confirmation cycle. This ‘travel-planning’ paradigm has been the focus of dialogue system research for several years. In such cases it suffices to develop a transition network for paths through dialogue states to recognisable completion states. Even if the database which the system accesses is dynamic (i.e. the information recorded there changes), the complexity of the interactions needed to support task completion in such an interface is low in comparison to that required to interact with a mobile agent with its own perceptions, in a changing world. Consider an operator’s conversation with an autonomous mobile robot with perceptions in a environment which is constantly changing. Dialogues with such a device will be very different (see e.g. arguments in [1]) to those in the travel-planning paradigm. There will be no predictable course of events in the dialogues. The device itself may ‘need’ to communicate urgently with its operator. There may not be well-defined endpoints to conversations, and relevant objects may appear and disappear from the operating environment. In particular, different ‘threads’ of a conversation may need to be 1Wallenberg laboratory for research on Information Technology and Autonomous Systems (WITAS) Unmanned Aerial Vehicle, under construction at Linköping University, Sweden. This research was funded under the WITAS Project by the Wallenberg Foundation, Sweden. initiated, set aside, and revisited, and operator and robot will need to negotiate the robot’s abilities in any situation. Tasks given to the device will also need to be specified, ordered, and their execution monitored. The dialogue modelling and management techniques developed under the travel-planning paradigm are not rich enough to support these more complex interaction scenarios, and we have found that different structures and methods need to be adopted. We discuss modelling and implementation of structures supporting these more complex conversational capabilities in in Sections 3 and 4.