Enabling the mobile web through auto-generating multimodal web pages

Multimodal interaction refers to interactions involving a combination of voice, text, stylus etc. Such multimodal interaction can provide advantages for mobile device input/output given the limitations of mobile device keypads and screens and the limits on user's availability to use such traditional inputs and outputs while mobile. One important resource that can be accessed from mobile devices is the World Wide Web, but Web pages, as they are visual and may require typed input, may not be readily suited to mobile device access. While there are technology initiatives addressing the authoring of new Web pages that are multimodal-enabled, the generating of multimodal versions of existing Web pages has not been addressed. To help address this problem, we propose in this paper an architecture and approach to autogenerating multimodal representations of existing Web pages, thereby helping to enable the Mobile Web.

[1]  Pedro M. Domingos,et al.  Adaptive Web Navigation for Wireless Devices , 2001, IJCAI.

[2]  Wolfgang Wahlster,et al.  SmartWeb: Mobile Applications of the Semantic Web , 2004, GI Jahrestagung.

[3]  Sharon L. Oviatt,et al.  Unification-based Multimodal Integration , 1997, ACL.

[4]  Jimmy J. Lin,et al.  Comparative Evaluation of a Natural Language Dialog Based System and a Menu Driven System for Information Access: a Case Study , 2000, RIAO.

[5]  Ivan Marsic,et al.  A framework for rapid development of multimodal interfaces , 2003, ICMI '03.

[6]  David Reitter,et al.  UI on the Fly: Generating a Multimodal User Interface , 2004, HLT-NAACL.

[7]  Xing Xie,et al.  Adapting Web pages for small-screen devices , 2005, IEEE Internet Computing.

[8]  Tharam S. Dillon,et al.  Mobile Web services discovery and invocation through auto-generation of abstract multimodal interface , 2005, International Conference on Information Technology: Coding and Computing (ITCC'05) - Volume II.

[9]  Philip R. Cohen,et al.  A map-based system using speech and 3D gestures for pervasive computing , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[10]  Mohammed Yeasin,et al.  A real-time framework for natural multimodal interaction with large screen displays , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[11]  Marilyn A. Walker,et al.  MATCH: An Architecture for Multimodal Dialogue Systems , 2002, ACL.

[12]  Sharon L. Oviatt,et al.  The efficiency of multimodal interaction: a case study , 1998, ICSLP.

[13]  Rainer Simon,et al.  Tool-supported single authoring for device independence and multimodality , 2005, Mobile HCI.