Bridging Web Applications and Perceptual User Interfaces

We present a system that bridges the perceptual user inter- face paradigm and web applications, through automatic analysis of the user interface layout. Perceptual input is mapped to application elements automatically by a new approach, the "Interface Robot", thus dramati- cally reducing the eort to adapt existing applications. In addition, we demonstrate how knowledge about the user interface provides a power- ful, yet easy to obtain, constraint to improve robustness of perceptual interpretation. First evaluation results for the approach are given with respect to a prototypical web application using hand-gestures.

[1]  Jesse James Garrett.  Ajax: A New Approach to Web Applications , 2007 .

[2]  Claudio S. Pinhanez,et al.  Interacting with steerable projected displays , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[3]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[4]  Claudio S. Pinhanez,et al.  Dynamically reconfigurable vision-based user interfaces , 2004, Machine Vision and Applications.

[5]  Sharon L. Oviatt,et al.  Ten myths of multimodal interaction , 1999, Commun. ACM.

[6]  Matthew Turk,et al.  Perceptual user interfaces (introduction) , 2000, CACM.

[7]  Frank Lömker,et al.  Building Modular Vision Systems with a Graphical Plugin Environment , 2006, Fourth IEEE International Conference on Computer Vision Systems (ICVS'06).

[8]  James L. Crowley,et al.  User-Centric Design of a Vision System for Interactive Applications , 2006, Fourth IEEE International Conference on Computer Vision Systems (ICVS'06).