A Multimodal Speech Interface for Accessing Web Pages

We present an interface for multimodal access to Web pages for German newspapers which integrates spoken and written input, as well as point and click operations and discuss the motivations behind it. As with many systems being developed recently, the speech modality is the main focus of our research. Our system shows new ways of integrating speech and language with classical access methods, and investigates the respective shortcomings and advantages of different combinations. The innovation lies specifically in two areas: the possibility for the user to refer to the content of pages, and the real integration of semantic content from different modalities. This paper also presents partial results of the project, as well as a fairly detailed analysis of the system’s components.