Agora: a GUI approach to multimodal user interfaces

Multi-modal user interfaces bring the best of the two worlds to the user; the navigational aspect of graphical user interfaces and the declarative aspect of voice interfaces. The user can move around forms providing a concrete visual representation, while speech input allows natural form filling and form navigation. Unlike the unimodal speech interfaces, the visual interface efficiently reflects the state of the dialog as well as the speech recognition and understanding results. Agora uses a hierarchical form flow motivated from the traditional GUI design philosophy and is loosely based on VoiceXML. Agora also brings up several design issues including: ambiguity resolution, modality synchronization, focus representation, preferred modality for input and output.