JVoiceXML as a modality component in the W3C multimodal architecture

Research regarding multimodal interaction led to a multitude of proposals for suitable software architectures. With all architectures describing multimodal systems differently, interoperability is severely hindered. The W3C MMI architecture is a proposed recommendation for a common architecture. In this article, we describe our experiences integrating JVoiceXML into the W3C MMI architecture and identify general limitations with regard to the available design space.

[1]  Sharon Oviatt,et al.  Multimodal Interfaces , 2008, Encyclopedia of Multimedia.

[2]  Roope Raisamo,et al.  Multimodal human-computer interaction: a constructive and empirical study , 1999 .

[3]  Gérard Bailly,et al.  Audiovisual Speech Synthesis , 2003, Int. J. Speech Technol..

[4]  Joëlle Coutaz,et al.  A design space for multimodal systems: concurrent processing and data fusion , 1993, INTERCHI.

[5]  Sangkyu Park,et al.  Multimodal user interfaces in the Open Agent Architecture , 1997, IUI '97.

[6]  Philippe A. Palanque,et al.  Fusion engines for multimodal input: a survey , 2009, ICMI-MLMI '09.

[7]  Stefan Kopp,et al.  The Behavior Markup Language: Recent Developments and Challenges , 2007, IVA.

[8]  Raj Srinivasan,et al.  RPC: Remote Procedure Call Protocol Specification Version 2 , 1995, RFC.

[9]  Nathaniel Palmer,et al.  Workflow Management Coalition , 2009, Encyclopedia of Database Systems.

[10]  Markku Turunen,et al.  An architecture and applications for speech-based accessibility systems , 2005, IBM Syst. J..

[11]  Fabio Paternò,et al.  MARIA: A universal, declarative, multiple abstraction-level language for service-oriented applications in ubiquitous environments , 2009, TCHI.

[12]  Douglas B. Moran,et al.  The Open Agent Architecture: A Framework for Building Distributed Software Systems , 1999, Appl. Artif. Intell..

[13]  Sharon L. Oviatt,et al.  Multimodal Interfaces: A Survey of Principles, Models and Frameworks , 2009, Human Machine Interaction.

[14]  Tsuneo Nitta,et al.  XISL: a language for describing multimodal interaction scenarios , 2003, ICMI '03.

[15]  David Harel,et al.  Modeling Reactive Systems With Statecharts : The Statemate Approach , 1998 .

[16]  Sharon L. Oviatt,et al.  Advances in Robust Multimodal Interface Design , 2003, IEEE Computer Graphics and Applications.

[17]  S. McCarron,et al.  XHTML™ 1.1 - Module-based XHTML , 2001 .

[18]  Max Mühlhäuser,et al.  MundoCore: A light-weight infrastructure for pervasive computing , 2007, Pervasive Mob. Comput..

[19]  Morten Fjeld,et al.  BounceSlider: actuated sliders for music performance and composition , 2008, Tangible and Embedded Interaction.

[20]  Andreas Neumann.  Scalable Vector Graphics (SVG) , 2008, Encyclopedia of GIS.

[21]  E. Michael Maximilien,et al.  Facts, trends and challenges in modern software development , 2012 .

[22]  Joëlle Coutaz,et al.  PAC, an Object Oriented Model for Dialog Design , 1987 .

[23]  A BoltRichard,et al.  Put-that-there , 1980 .

[24]  Frank Vahid,et al.  Enabling nonexpert construction of basic sensor-based systems , 2009, TCHI.

[25]  Richard A. Bolt,et al.  “Put-that-there”: Voice and gesture at the graphics interface , 1980, SIGGRAPH '80.

[26]  Keiichi Tokuda,et al.  Galatea: Open-Source Software for Developing Anthropomorphic Spoken Dialog Agents , 2004, Life-like characters.

[27]  Wolfgang Wahlster,et al.  Readings in Intelligent User Interfaces , 1998 .

[28]  david duce,et al.  Scalable Vector Graphics (SVG): Vector Graphics for the Web , 2001 .

[29]  Stéphane Chatty,et al.  Extending a graphical toolkit for two-handed interaction , 1994, UIST '94.

[30]  Arjeh M. Cohen,et al.  Synchronized Multimedia Integration Language (SMIL) 2.0 , 1998 .

[31]  Constantinos Phanouriou,et al.  UIML: A Device-Independent User Interface Markup Language , 2000 .