Software Architecture for Multimodal User Input - FLUID

Traditional ways to handle user input in software are uncomfortable when an application wishes to use novel input devices. This is especially the case in gesture based user interfaces. In this paper we describe these problems and as a solution we present an architecture and an implementation of a user input toolkit. We show that the higher level processing of user input such as gesture recognition requires a whole newkind of paradigm. The system we designed and implemented - FLexible User Input Design (FLUID) - is a lightweight library that can be used in different kinds of software. The potential application areas include all system where novel input devices are in use: virtual reality, entertainment systems and embedded systems.

[1]  Joëlle Coutaz,et al.  A design space for multimodal systems: concurrent processing and data fusion , 1993, INTERCHI.

[2]  Carolina Cruz-Neira,et al.  VR Juggler: a virtual platform for virtual reality application development , 2001, Proceedings IEEE Virtual Reality 2001.

[3]  James A. Landay,et al.  Sketching Interfaces: Toward More Human Interface Design , 2001, Computer.

[4]  Gregory D. Abowd,et al.  The context toolkit: aiding the development of context-enabled applications , 1999, CHI '99.

[5]  Sharon L. Oviatt,et al.  Multimodal Interaction for 2D and 3D Environments , 1999, IEEE Computer Graphics and Applications.

[6]  Lambert Schomaker,et al.  A taxonomy of Multimodal Interaction in the Human Information Processing System , 1995 .

[7]  Russell M. Taylor,et al.  VRPN: a device-independent, network-transparent VR peripheral system , 2001, VRST '01.

[8]  Tommi Ilmonen,et al.  Accelerometer-based motion tracking for orchestra conductor following , 2000 .

[9]  Tommi Ilmonen.  Tracking Conductor of an Orchestra Using Artificial Neural Networks , 1999 .

[10]  Oliver Bimber,et al.  A multi-layered architecture for sketch-based interaction within virtual environments , 2000, Comput. Graph..

[11]  Tapio Takala,et al.  Conductor Following With Artificial Neural Networks , 1999, ICMC.

[12]  Dieter Schmalstieg,et al.  An open software architecture for virtual reality interaction , 2001, VRST '01.

[13]  John G. Proakis,et al.  Digital Signal Processing: Principles, Algorithms, and Applications , 1992 .

[14]  Tommi Ilmonen.  Mustajuuri - An application and toolkit for interactive audio processing , 2001 .

[15]  John G. Proakis,et al.  Digital signal processing (2nd ed.): principles, algorithms, and applications , 1992 .

[16]  Gregory D. Abowd,et al.  Providing integrated toolkit-level support for ambiguity in recognition-based interfaces , 2000, CHI.

[17]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[18]  T. Ilmonen.  Immersive 3d User Interface for Computer Animation Control , 2002 .