Facial feature tracking for cursor control

This work is motivated by the goal of providing a non-contact means of controlling the mouse pointer on a computer system for people with motor difficulties using low-cost, widely available hardware. The required information is derived from video data captured using a web camera mounted below the computer's monitor. A colour filter is used to identify skin coloured regions. False positives are eliminated by optionally removing background regions and by applying statistical rules that reliably identify the largest skin-coloured region, which is assumed to be the user's face. The nostrils are then found using heuristic rules. The instantaneous location of the nostrils is compared with their at-rest location; any significant displacement is used to control the mouse pointer's movement. The system is able to process 18 frames per second at a resolution of 320 by 240 pixels, or 30 fps at 160 by 120 pixels using moderately powerful hardware (a 500 MHz Pentium III desktop computer).

[1]  Stephen J. McKenna,et al.  Tracking interacting people , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[2]  M. Ibrahim Sezan,et al.  Face Tracking Using Robust Statistical Estimation , 1998 .

[3]  Shaogang Gong,et al.  Tracking faces , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[4]  P Blenkhorn,et al.  Controlling mouse pointer position using an infrared head-operated joystick. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[5]  Roberto Brunelli,et al.  Face Recognition: Features Versus Templates , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Alexander H. Waibel,et al.  Skin-Color Modeling and Adaptation , 1998, ACCV.

[7]  Alexander H. Waibel,et al.  A real-time face tracker , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[8]  Witold Pedrycz,et al.  Face recognition: A study in information fusion using fuzzy integral , 2005, Pattern Recognit. Lett..

[9]  Alan L. Yuille,et al.  Feature extraction from faces using deformable templates , 2004, International Journal of Computer Vision.

[10]  David R. Bull,et al.  Robust H.263+ video for real-time Internet applications , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[11]  Paul Blenkhorn,et al.  Blink detection for real-time eye tracking , 2002, J. Netw. Comput. Appl..

[12]  A. Ardeshir Goshtasby,et al.  Detecting human faces in color images , 1998, Image Vis. Comput..

[13]  Ioannis Pitas,et al.  Facial feature extraction and pose determination , 2000, Pattern Recognit..

[14]  Alex Waibel,et al.  Gaze Tracking Based on Face‐Color , 1995 .

[15]  D. G. Evans,et al.  A head operated “joystick” , 1996 .

[16]  J. M. Stroud.  THE FINE STRUCTURE OF PSYCHOLOGICAL TIME , 1967 .

[17]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[18]  Bernd Menser,et al.  Segmentation of human faces in color images using connected operators , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[19]  George C. Stockman,et al.  Menu Selection by Facial Aspect , 1999 .

[20]  Kentaro Toyama,et al.  Manipulation of Video Eye Gaze and Head Orientation for Video Teleconferencing , 1999 .

[21]  Ioannis Pitas,et al.  Extraction of facial regions and features using color and shape information , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[22]  E. Granum,et al.  Skin colour detection under changing lighting conditions , 1999 .

[23]  Ioannis Pitas,et al.  Segmentation and tracking of faces in color images , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[24]  Kentaro Toyama,et al.  “Look, Ma – No Hands!” Hands-Free Cursor Control with Real-Time 3D Face Tracking , 1998 .

[25]  Paul Blenkhorn,et al.  A Head Operated Joystick--Experience with Use. , 1999 .

[26]  Alex Pentland,et al.  Computer Vision for Human–Machine Interaction: Acknowledgements , 1998 .