Face as mouse through visual face tracking

This paper introduces a novel camera mouse driven by visual face tracking based on a 3D model. As the camera becomes standard configuration for personal computers (PCs) and computation speed increases, achieving human-machine interaction through visual face tracking becomes a feasible solution to hands-free control. Human facial movements can be broken down into rigid motions, such as rotation and translation, and non-rigid motions such as opening, closing, and stretching of the mouth. First, we describe our face tracking system which can robustly and accurately retrieve these motion parameters from videos in real time [H. Tao, T. Huang, Explanation-based facial motion tracking using a piecewise Bezier volume deformation model, in: Proceedings of IEEE Computer Vision and Pattern Recogintion, vol. 1, 1999, pp. 611-617]. The retrieved (rigid) motion parameters can be employed to navigate the mouse cursor; the detection of mouth (non-rigid) motions triggers mouse events in the operating system. Three mouse control modes are investigated and their usability is compared. Experiments in the Windows XP environment verify the convenience of our camera mouse in hands-free control. This technology can be an alternative input option for people with hand and speech disability, as well as for futuristic vision-based games and interfaces.

[1]  Robert J. K. Jacob,et al.  What you look at is what you get: eye movement-based interaction techniques , 1990, CHI '90.

[2]  Kentaro Toyama,et al.  “Look, Ma – No Hands!” Hands-Free Cursor Control with Real-Time 3D Face Tracking , 1998 .

[3]  Dmitry O. Gorodnichy,et al.  On importance of nose for face tracking , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[4]  Dmitry O. Gorodnichy.  Nouse ‘Use Your Nose as a Mouse’ – a New Technology for Hands-free Games and Interfaces , 2002 .

[5]  M. Betke,et al.  The Camera Mouse: visual tracking of body features to provide computer access for people with severe disabilities , 2002, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[6]  Timothy F. Cootes,et al.  Active Shape Model Search using Local Grey-Level Models: A Quantitative Evaluation , 1993, BMVC.

[7]  Thomas S. Huang,et al.  Explanation-based facial motion tracking using a piecewise Bezier volume deformation model , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[8]  Paul A. Viola,et al.  Fast and Robust Classification using Asymmetric AdaBoost and a Detector Cascade , 2001, NIPS.

[9]  Matthew Turk,et al.  Perceptual user interfaces , 2000 .

[10]  Matthew Turk,et al.  Perceptual user interfaces (introduction) , 2000, CACM.

[11]  Gregory D. Abowd,et al.  Perceptual user interfaces using vision-based eye tracking , 2003, ICMI '03.

[12]  James W. Davis,et al.  A perceptual user interface for recognizing head gesture acknowledgements , 2001, PUI '01.

[13]  Robert S. Olyha,et al.  Negative inertia: a dynamic pointing function , 1995, CHI '95.

[14]  Thomas S. Huang,et al.  Face localization via hierarchical CONDENSATION with Fisher boosting feature selection , 2004, CVPR 2004.