Staging transformations for multimodal web interaction management

Multimodal interfaces are becoming increasingly ubiquitous with the advent of mobile devices, accessibility considerations, and novel software technologies that combine diverse interaction media. In addition to improving access and delivery capabilities, such interfaces enable flexible and personalized dialogs with websites, much like a conversation between humans. In this paper, we present a software framework for multimodal web interaction management that supports mixed-initiative dialogs between users and websites. A mixed-initiative dialog is one where the user and the website take turns changing the flow of interaction. The framework supports the functional specification and realization of such dialogs using staging transformations -- a theory for representing and reasoning about dialogs based on partial input. It supports multiple interaction interfaces, and offers sessioning, caching, and co-ordination functions through the use of an interaction manager. Two case studies are presented to illustrate the promise of this approach.

[1]  Doron Cohen,et al.  Personalized pocket directories for mobile devices , 2002, WWW.

[2]  Neel Sundaresan,et al.  AVoN calling: AXL for voice-enabled Web navigation , 2000, Comput. Networks.

[3]  Naren Ramakrishnan,et al.  Personalizing interactions with information systems , 2003, Adv. Comput..

[4]  Kuansan Wang.  A Study of Semantics Synchronous Understanding on Speech Interface Design , 2003 .

[5]  Baoyao Zhou,et al.  Function-based object model towards website adaptation , 2001, WWW '01.

[6]  Nicole Yankelovich,et al.  How do users know what to say? , 1996, INTR.

[7]  Joseph Pasquale,et al.  A web middleware architecture for dynamic customization of content for wireless clients , 2002, WWW '02.

[8]  In-Young Ko,et al.  Dynamic coordination of information management services for processing dynamic web content , 2002, WWW '02.

[9]  Jaideep Srivastava,et al.  Automatic personalization based on Web usage mining , 2000, CACM.

[10]  David R. Karger,et al.  User interface continuations , 2003, UIST '03.

[11]  Alistair N. Coles,et al.  A framework for coordinated multi-modal browsing with multiple clients , 2003, WWW '03.

[12]  David W. Binkley,et al.  Interprocedural slicing using dependence graphs , 1990, TOPL.

[13]  Peter Brusilovsky,et al.  Adaptive hypermedia: from systems to framework , 1999, CSUR.

[14]  David Frohlich,et al.  MIXED INITIATIVE INTERACTION , 1991 .

[15]  James Veitch.  A conversation Paul Graham , 1998, CACM.

[16]  Thomas Ball,et al.  Mawl: A Domain-Specific Language for Form-Based Services , 1999, IEEE Trans. Software Eng..

[17]  Marti A. Hearst,et al.  Finding the flow in web site search , 2002, CACM.

[18]  Jen-Yao Chung,et al.  Web services computing: advancing software interoperability , 2003, Computer.

[19]  Giovanni Maria Sacco,et al.  Dynamic Taxonomies: A Model for Large Information Bases , 2000, IEEE Trans. Knowl. Data Eng..

[20]  David W. Binkley,et al.  Interprocedural slicing using dependence graphs , 1988, SIGP.

[21]  Christian Queinnec,et al.  The influence of browsers on evaluators or, continuations to program web servers , 2000, ICFP '00.

[22]  David W. Binkley,et al.  Program slicing , 2008, 2008 Frontiers of Software Maintenance.

[23]  Manuel A. Pérez-Quiñones,et al.  WebContext: remote access to shared context , 2001, PUI '01.

[24]  Matthias Felleisen,et al.  Automatically Restructuring Programs for the Web , 2004, Automated Software Engineering.

[25]  Thomas Rist,et al.  From adaptive hypertext to personalized web companions , 2002, CACM.

[26]  Jennifer C. Lai,et al.  Conversational interfaces , 2000, CACM.

[27]  Manuel A. Pérez-Quiñones,et al.  The Staging Transformation Approach to Mixing Initiative , 2003, IJCAI 2003.

[28]  Andreas Paepcke,et al.  Seeing the whole in parts: text summarization for web browsing on handheld devices , 2001, WWW '01.

[29]  YankelovichNicole.  How do users know what to say , 1996 .

[30]  Michael Benedikt,et al.  Experience with a Domain Specific Language for Form-based Services , 1997, DSL.

[31]  Gary Marchionini,et al.  Information Seeking in Electronic Environments , 1995 .

[32]  Naren Ramakrishnan,et al.  Personalizing Web sites with mixed-initiative interaction , 2003 .

[33]  Savitha Srinivasan,et al.  Is Speech Recognition Becoming Mainstream? , 2002, Computer.

[34]  Brad A. Myers,et al.  Guest Editors' Introduction: Handheld Computing , 2003, Computer.

[35]  David Carmel,et al.  Knowledge encapsulation for focused search from pervasive devices , 2002, TOIS.

[36]  Curry Guinn,et al.  Mixed-initiative interaction , 1999 .

[37]  Mary Beth Rosson,et al.  Taking the Initiative with Extempore: Exploring Out-of-Turn Interactions with Websites , 2003, ArXiv.

[38]  Juliana Freire,et al.  WebViews: accessing personalized web content and services , 2001, WWW '01.

[39]  Neil D. Jones,et al.  An introduction to partial evaluation , 1996, CSUR.

[40]  Pedro M. Domingos,et al.  Mixed initiative interfaces for learning tasks: SMARTedit talks back , 2001, IUI '01.

[41]  Nicholas J. Belkin,et al.  Cases, scripts, and information-seeking strategies: On the design of interactive information retrieval systems , 1995 .

[42]  Wei-Ying Ma,et al.  Detecting web page structure for adaptive viewing on small form factor devices , 2003, WWW '03.

[43]  Candace L. Sidner,et al.  Attention, Intentions, and the Structure of Discourse , 1986, CL.

[44]  Peter Brusilovsky,et al.  Adaptive Hypermedia , 2001, User Modeling and User-Adapted Interaction.

[45]  Austin Henderson,et al.  Design: static pages are dead: how a modular approach is changing interaction design , 2001, INTR.

[46]  Allen Newell,et al.  Computer text-editing: An information-processing analysis of a routine cognitive skill , 1980, Cognitive Psychology.

[47]  Claus Brabrand,et al.  The bigwig Project , 2000 .

[48]  Terry Winograd,et al.  Understanding computers and cognition - a new foundation for design , 1987 .