MundoMonkey: customizing interaction with web applications in interactive spaces

We notice an increasing usage of web applications in interactive spaces, a variant of ubiquitous computing environments. Interactive spaces feature a large and dynamically changing number of devices, e.g., an interactive TV set in the living room that is used with different input devices or an iPhone that is dynamically federated to devices in the environment. Web applications need a better way to exploit the resources in the interactive space beyond the standard input devices like mouse and keyboard, e.g., a speech recognition device. This paper presents MundoMonkey a web browser extension and programming API for interactive spaces. The API follows the event based programming paradigm for allowing web applications and end-user scripts to access the interactive space. Our approach aligns well with the commonly used programming style for web applications. We used MundoMonkey to customize the interface of web applications to user preferences and the interactive space at hand. To our knowledge our approach is the first to address adaptation of the output as well as processing of input data. With MundoMonkey the customization is performed transparently to the application developer by the end-user. Thereby, MundoMonkey is an alternative to model driven user interface development approaches.

[1]  Joëlle Coutaz,et al.  Coupling interaction resources: an analytical model , 2005, sOc-EUSAI '05.

[2]  Caroline Appert,et al.  SwingStates: adding state machines to the swing toolkit , 2006, UIST.

[3]  Jean-Yves Lionel Lawson,et al.  The openinterface framework: a tool for multimodal interaction. , 2008, CHI Extended Abstracts.

[4]  Eben M. Haber,et al.  CoScripter: automating & sharing how-to knowledge in the enterprise , 2008, CHI.

[5]  Brad A. Myers,et al.  Using handhelds and PCs together , 2001, CACM.

[6]  Jean-Sébastien Sottet,et al.  An MDE-SOA Approach to Support Plastic User Interfaces in Ambient Spaces , 2007, HCI.

[7]  Pierre Dragicevic,et al.  Support for input adaptability in the ICON toolkit , 2004, ICMI '04.

[8]  Meredith Ringel Morris,et al.  iStuff: a physical user interface toolkit for ubiquitous computing environments , 2003, CHI '03.

[9]  Armando Fox,et al.  The Interactive Workspaces Project: Experiences with Ubiquitous Computing Rooms , 2002, IEEE Pervasive Comput..

[10]  Richard E. Ladner,et al.  Accessmonkey: a collaborative scripting framework for web users and developers , 2007, W4A '07.

[11]  Stephen W. Gilroy,et al.  Using Interaction Style to Match the Ubiquitous User Interface to the Device-to-Hand , 2004, EHCI/DS-VIS.

[12]  Jingtao Wang,et al.  Theoretical and architectural support for input device adaptation , 2002 .

[13]  Fabio Paternò,et al.  TERESA: a transformation-based environment for designing and developing multi-device interfaces , 2004, CHI EA '04.

[14]  Max Mühlhäuser,et al.  MundoCore: A light-weight infrastructure for pervasive computing , 2007, Pervasive Mob. Comput..

[15]  Karin Coninx,et al.  Towards model-based design support for distributed user interfaces , 2004, NordiCHI '04.

[16]  Krzysztof Z. Gajos,et al.  Automatically generating user interfaces adapted to users' motor and vision capabilities , 2007, UIST.

[17]  Melanie Hartmann,et al.  Association: Unobtrusively Creating Digital Contracts with Smart Products , 2008 .

[18]  Armando Fox,et al.  The Event Heap: a coordination infrastructure for interactive workspaces , 2002, Proceedings Fourth IEEE Workshop on Mobile Computing Systems and Applications.

[19]  Scott R. Klemmer,et al.  Programming by a sample: rapidly creating web applications with d.mix , 2007, UIST.

[20]  Armando Fox,et al.  Patch panel: enabling control-flow interoperability in ubicomp environments , 2004, Second IEEE Annual Conference on Pervasive Computing and Communications, 2004. Proceedings of the.

[21]  Alan F. Blackwell,et al.  Dasher—a data entry interface using continuous gestures and language models , 2000, UIST '00.

[22]  Gregory D. Abowd,et al.  Providing integrated toolkit-level support for ambiguity in recognition-based interfaces , 2000, CHI.

[23]  Kris Luyten,et al.  ReWiRe: Designing Reactive Systems for Pervasive Environments , 2008, DSV-IS.