Perceptive User Interface, a Generic Approach

This paper describes the development of a real-time perceptive user interface. Two cameras are used to detect a user's head, eyes, hand, fingers and gestures. These cues are interpreted to control a user interface on a large screen. The result is a fully functional integrated system that processes roughly 7.5 frames per second on a Pentium IV system. The calibration of this setup is carried out through a few simple and intuitive routines, making the system adaptive and accessible to non-expert users. The minimal hardware requirements are two web-cams and a computer. The paper will describe how the user is observed (head, eye, hand and finger detection, gesture recognition), the 3D geometry involved, and the calibration steps necessary to set up the system.

[1]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[2]  Wim Westera,et al.  The Design of the Virtual Company: Synergism of Learning and Working in a Networked Environment , 2000 .

[3]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[4]  R. Plankers,et al.  Articulated soft objects for video-based body modeling , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[5]  Luc Van Gool,et al.  Real-time range scanning of deformable surfaces by adaptively coded structured light , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[6]  Yi-Ping Hung,et al.  Free-hand pointer by use of an active stereo vision system , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[7]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[8]  Mario Hernández-Tejera,et al.  Hand Gesture Recognition for Human-Machine Interaction , 2004, WSCG.

[9]  Thomas P. Koninckx,et al.  High-speed active 3D acquisition based on a pattern-specific mesh , 2003, IS&T/SPIE Electronic Imaging.

[10]  Bruno Jedynak,et al.  Maximum Entropy Models for Skin Detection , 2003, EMMCVPR.

[11]  James M. Rehg,et al.  Statistical Color Models with Application to Skin Detection , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[12]  Til Aach,et al.  Illumination-Invariant Change Detection Using a Statistical Colinearity Criterion , 2001, DAGM-Symposium.

[13]  Diana Laurillard,et al.  Rethinking University Teaching: A Conversational Framework for the Effective Use of Learning Technologies. 2nd Edition , 1993 .

[14]  Peter Sloep.  Learning Objects: The Answer to the Knowledge Economy's Predicament? , 2004 .