Hands-free vision-based interface for computer accessibility

Physically disabled and mentally challenged people are an important part of our society that has not yet received the same opportunities as others in their inclusion in the Information Society. Therefore, it is necessary to develop easily accessible systems for computers to achieve their inclusion within the new technologies. This paper presents a project whose objective is to draw disabled people nearer to new technologies. It presents a vision-based user interface designed to achieve computer accessibility for disabled users with motor impairments. The interface automatically finds the user's face and tracks it through time to recognize gestures within the face region in real time. Subsequently, a new information fusion procedure is proposed to acquire data from computer vision algorithms and its results are used to carry out a robust recognition process. Finally, we show how the system is used to replace a conventional mouse device for computer interaction and as a communication system for non-verbal children.

[1]  Giuseppe Polese,et al.  Iconic language design for people with significant speech and multiple impairments , 1994, Assistive Technology and Artificial Intelligence.

[2]  Gary R. Bradski,et al.  Real time face and object tracking as a component of a perceptual user interface , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[3]  Paul Blenkhorn,et al.  Blink detection for real-time eye tracking , 2002, J. Netw. Comput. Appl..

[4]  Jing Xiao,et al.  Robust full-motion recovery of head by dynamic templates and re-registration techniques , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[5]  Antonio Otero,et al.  Exposiciones interactivas, caso de estudio: Galicia Dixital , 2007 .

[6]  Margrit Betke,et al.  Evaluation of tracking methods for human-computer interaction , 2002, Sixth IEEE Workshop on Applications of Computer Vision, 2002. (WACV 2002). Proceedings..

[7]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Tim Morris,et al.  Facial feature tracking for cursor control , 2006, J. Netw. Comput. Appl..

[9]  Margrit Betke,et al.  Communication via eye blinks and eyebrow raises: video-based human-computer interfaces , 2003, Universal Access in the Information Society.

[10]  Lakhmi C. Jain,et al.  Network and information security: A computational intelligence approach: Special Issue of Journal of Network and Computer Applications , 2007, J. Netw. Comput. Appl..

[11]  Margrit Betke,et al.  Communication via eye blinks - detection and duration analysis in real time , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[12]  Dmitry O. Gorodnichy.  Nouse ‘Use Your Nose as a Mouse’ – a New Technology for Hands-free Games and Interfaces , 2002 .

[13]  Rick Kjeldsen,et al.  Improvements in vision-based pointer control , 2006, Assets '06.

[14]  Dorin Comaniciu,et al.  Kernel-Based Object Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Julio Abascal,et al.  Universal accessibility as a multimodal design issue , 2007, CACM.

[16]  Mathias Kölsch,et al.  Emerging Topics in Computer Vision , 2004 .

[17]  Thia Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation: Theory, Algorithms and Software , 2001 .

[18]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[19]  Cristina Manresa-Yee,et al.  Towards Hands-Free Interfaces Based on Real-Time Robust Facial Gesture Recognition , 2006, AMDO.

[20]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[21]  Jing Xiao,et al.  Robust full‐motion recovery of head by dynamic templates and re‐registration techniques , 2003 .

[22]  Kentaro Toyama,et al.  “Look, Ma – No Hands!” Hands-Free Cursor Control with Real-Time 3D Face Tracking , 1998 .

[23]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[24]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[25]  M. Betke,et al.  The Camera Mouse: visual tracking of body features to provide computer access for people with severe disabilities , 2002, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[26]  Dmitry O. Gorodnichy,et al.  Nouse 'use your nose as a mouse' perceptual vision technology for hands-free games and interfaces , 2004, Image Vis. Comput..

[27]  L. Rothkrantz,et al.  Toward an affect-sensitive multimodal human-computer interaction , 2003, Proc. IEEE.

[28]  Klaus J. Kirchberg,et al.  Robust Face Detection Using the Hausdorff Distance , 2001, AVBPA.

[29]  C. Manresa-Yee,et al.  Non-verbal communication by means of head tracking , 2006 .

[30]  Bernard F. Buxton,et al.  Statistical Modeling of Colour Data , 2004, International Journal of Computer Vision.

[31]  Gerhard Roth,et al.  Nouse 'Use Your Nose as a Mouse' - a New Technology for Hands-free Games and Interfaces , 2002 .