“Look, Ma – No Hands!” Hands-Free Cursor Control with Real-Time 3D Face Tracking

This paper presents a real-time, non-invasive, vision system which performs hands-free cursor control by having users point their nose where they wish to place the cursor on a monitor screen. The vision system robustly tracks 3D face position and orientation in real time using a framework called Incremental Focus of Attention (IFA). IFA integrates tracking based on multiple cues (including color, intensity templates, and dark point features) which cooperate to track under adverse visual conditions. The pose recovered from tracking is then used to compute the intersection between the plane of the monitor screen and an imaginary ray extending forward from the user’s nose. Results show that naive users can position a cursor within a 1cm x 1cm square from a distance of 50cm from the monitor.

[1]  Thomas S. Huang,et al.  3D head pose computation from 2D images: templates versus features , 1995, Proceedings., International Conference on Image Processing.

[2]  Alex Pentland,et al.  Parametrized structure from motion for 3D adaptive feedback tracking of faces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Alex Pentland,et al.  LAFTER: lips and face real time tracker , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Alex Waibel,et al.  Tracking Eyes and Monitoring Eye Gaze , 1997 .

[5]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[6]  Roberto Cipolla,et al.  Fast visual tracking by temporal consensus , 1996, Image Vis. Comput..

[7]  Matthew Turk,et al.  Visual interaction with lifelike characters , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[8]  Keith Duncan,et al.  Cognitive Engineering , 2017, Encyclopedia of GIS.

[9]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[10]  Tomaso A. Poggio,et al.  Multidimensional morphable models , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[11]  Gregory D. Hager,et al.  Incremental focus of attention for robust visual tracking , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Atsuto Maki,et al.  Hyper-patches for 3D model acquisition and tracking , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Stanley T. Birchfield,et al.  Elliptical head tracking using intensity gradients and color histograms , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[14]  Gerasimos Potamianos,et al.  Speaker independent audio-visual database for bimodal ASR , 1997, AVSP.

[15]  George C. Stockman,et al.  Controlling a computer via facial aspect , 1995, IEEE Trans. Syst. Man Cybern..

[16]  Hans Peter Graf,et al.  Robust face feature analysis for automatic speechreading and character animation , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[17]  Robert M. Haralick,et al.  Determining camera parameters from the perspective projection of a rectangle , 1989, Pattern Recognit..

[18]  Pamela J. Fink,et al.  If at First You Don’t Succeed , 1993 .

[19]  Laurent D. Cohen,et al.  On active contour models and balloons , 1991, CVGIP Image Underst..