A dialogue agent for navigation support in virtual reality

We describe our work on designing a natural language accessible navigation agent for a virtual reality (VR) environment. The agent is part of an agent framework, which means that it can communicate with other agents. Its navigation task consists of guiding the visitors in the environment and to answer questions about this environment (a theatre building). Visitors are invited to explore this building, see what is there, ask questions and get advice from the navigation agent. A 2D map has been added to the environment so that visitors can make references to the locations and objects on this map, both in natural language and by clicking with the mouse, making it a multimodal system with cross-modality references.