Examining the feasibility of face gesture detection using a wheelchair mounted camera

he user interface of existing autonomous wheelchairs concentrates on direct control of the wheelchair by the user using mechanical devices or various hand, head or face gestures. However, it is important to monitor the user to ensure safety an comfort of the user, who operates the autonomous wheelchair. In addition, such monitoring of a user greatly improves usablity of an autonomous wheelchair due to the improved communication between the user and the wheelchair. This paper proposes a user monitoring system for an autonomous wheelchair. The feedback of the user and the information about the actions of the user, obtained by such a system, will be used by the autonomous wheelchair for planning of its future actions. As a first step towards creation of the monitoring system, this work proposes and examines the feasibility of a system that is capable of recognizing static facial gestures of the user using a camera mounted on a wheelchair. The prototype of such a system has been implemented and tested, achieving 90% recognition rate with 6% false positive and 4% false negative rates.

[1]  Timothy F. Cootes,et al.  Interpreting face images using active appearance models , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[2]  Sébastien Marcel,et al.  Hand gesture recognition using input-output hidden Markov models , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[3]  Katsuhiko Sakaue,et al.  Stereo Camera Based Non-Contact Non-Constraining Head Gesture Interface for Electric Wheelchairs , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[4]  Myung Jin Chung,et al.  Extension of cascaded simple feature based face detection to facial expression recognition , 2008, Pattern Recognit. Lett..

[5]  Christian Bauckhage,et al.  Fast learning for customizable head pose recognition in robotic wheelchair control , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[6]  Shinjiro Kawato,et al.  Real-time detection of nodding and head-shaking by directly detecting and tracking the "between-eyes" , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[7]  Huosheng Hu,et al.  Head gesture recognition for hands-free control of an intelligent wheelchair , 2007, Ind. Robot.

[8]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[9]  Ho-Sub Yoon,et al.  Hand gesture recognition using combined features of location, angle and velocity , 2001, Pattern Recognit..

[10]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[12]  George Awad,et al.  Real Time Hand Gesture Recognition Including Hand Segmentation and Tracking , 2006, ISVC.

[13]  Y. Matsumotot,et al.  Development of intelligent wheelchair system with face and gaze based interface , 2001, Proceedings 10th IEEE International Workshop on Robot and Human Interactive Communication. ROMAN 2001 (Cat. No.01TH8591).

[14]  Ashish Kapoor,et al.  A real-time head nod and shake detector , 2001, PUI '01.

[15]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[16]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[17]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Alfredo Gardel Vicente,et al.  Commands generation by face movements applied to the guidance of a wheelchair for handicapped people , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[19]  Manuel Mazo,et al.  Wheelchair Guidance Strategies Using EOG , 2002, J. Intell. Robotic Syst..

[20]  Guojun Lu,et al.  A Comparative Study of Fourier Descriptors for Shape Representation and Retrieval , 2002 .

[21]  Katsuhiko Sakaue,et al.  Development of head gesture interface for electric wheelchair , 2007, i-CREATe '07.

[22]  Attila Fazekas,et al.  Recognition of Facial Gestures Based on Support Vector Machines , 2005, IbPRIA.

[23]  Yoshiaki Shirai,et al.  Intelligent wheelchair using visual information on human faces , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[24]  R. Simpson.  Smart wheelchairs: A literature review. , 2005, Journal of rehabilitation research and development.

[25]  C. Goodall.  Procrustes methods in the statistical analysis of shape , 1991 .

[26]  Dimitris N. Metaxas,et al.  Emblem Detections by Tracking Facial Features , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[27]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[28]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[29]  Gwen Littlewort,et al.  Real Time Face Detection and Facial Expression Recognition: Development and Applications to Human Computer Interaction. , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[30]  Guojun Lu,et al.  A comparative study of curvature scale space and Fourier descriptors for shape-based image retrieval , 2003, J. Vis. Commun. Image Represent..

[31]  Jayantha Katupitiya,et al.  A hand gesture based virtual interface for wheelchair control , 2003, Proceedings 2003 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM 2003).

[32]  Yutaka Satoh,et al.  An Omnidirectional Stereo Vision-Based Smart Wheelchair , 2007, EURASIP J. Image Video Process..

[33]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[35]  Matti Pietikäinen,et al.  An Experimental Comparison of Autoregressive and Fourier-Based Descriptors in 2D Shape Classification , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Ralph Roskies,et al.  Fourier Descriptors for Plane Closed Curves , 1972, IEEE Transactions on Computers.

[38]  Caifeng Shan,et al.  Learning Discriminative LBP-Histogram Bins for Facial Expression Recognition , 2008, BMVC.

[39]  Mark de Berg,et al.  Computational geometry: algorithms and applications , 1997 .

[40]  Dennis Gabor,et al.  Theory of communication , 1946 .

[41]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[42]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[43]  Takashi Gomi,et al.  Developing Intelligent Wheelchairs for the Handicapped , 1998, Assistive Technology and Artificial Intelligence.

[44]  Karl-Friedrich Kraiss,et al.  Supervised navigation and manipulation for impaired wheelchair users , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[45]  Ying Wu,et al.  Vision-Based Gesture Recognition: A Review , 1999, Gesture Workshop.

[46]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[47]  Jeong-Su Han,et al.  Human-machine interface for wheelchair control with EMG and its evaluation , 2003, Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat. No.03CH37439).

[48]  Ari Visa,et al.  Enhanced Fourier Shape Descriptor Using Zero-Padding , 2005, SCIA.

[49]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[50]  N.D. Georganas,et al.  Real-time Vision-based Hand Gesture Recognition Using Haar-like Features , 2007, 2007 IEEE Instrumentation & Measurement Technology Conference IMTC 2007.

[51]  Yoshua Bengio,et al.  An Input Output HMM Architecture , 1994, NIPS.

[52]  Yoshiaki Shirai,et al.  Robotic wheelchair based on observations of both user and environment , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[53]  Peng Lu,et al.  Head Gesture Recognition Based on Bayesian Network , 2005, IbPRIA.

[54]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[55]  Holly A. Yanco.  Integrating Robotic Research: A Survey of Robotic Wheelchair Development , 1998 .

[56]  Jr. Joseph J. LaViola,et al.  A Survey of Hand Posture and Gesture Recognition Techniques and Technology , 1999 .

[57]  Toshiaki Ejima,et al.  Real-Time Hand Tracking and Gesture Recognition System , 2005 .

[58]  Matti Pietikäinen,et al.  A Visual System for Hand Gesture Recognition in Human-Computer Interaction , 2007, SCIA.

[59]  Ryohei Nakatsu,et al.  A Head Gesture Recognition Algorithm , 2000, ICMI.

[60]  H. Hotelling.  Analysis of a complex of statistical variables into principal components. , 1933 .

[61]  Dimitris N. Metaxas,et al.  Large Scale Learning of Active Shape Models , 2007, 2007 IEEE International Conference on Image Processing.

[62]  Chung-Lin Huang,et al.  Hand gesture recognition using a real-time tracking method and hidden Markov models , 2003, Image Vis. Comput..

[63]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[64]  Inhyuk Moon,et al.  Intelligent robotic wheelchair with EMG-, gesture-, and voice-based interfaces , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[65]  Mohan S. Kankanhalli,et al.  Shape Measures for Content Based Image Retrieval: A Comparison , 1997, Inf. Process. Manag..

[66]  Kazuo Tanaka,et al.  Electroencephalogram-based control of an electric wheelchair , 2005, IEEE Transactions on Robotics.

[67]  Takeo Kanade,et al.  Real-time combined 2D+3D active appearance models , 2004, CVPR 2004.

[68]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[69]  María-Elena Algorri,et al.  Facial gesture recognition for interactive applications , 2004, Proceedings of the Fifth Mexican International Conference in Computer Science, 2004. ENC 2004..

[70]  SeppänenTapio,et al.  An Experimental Comparison of Autoregressive and Fourier-Based Descriptors in 2D Shape Classification , 1995 .

[71]  Ari Visa,et al.  Multiscale Fourier descriptor for shape-based image retrieval , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[72]  Liyanage C. De Silva,et al.  Head gestures recognition , 2001, ICIP.

[73]  Tamás Szirányi,et al.  Dynamic training of hand gesture recognition system , 2004, ICPR 2004.

[74]  Chung-Lin Huang,et al.  A model-based hand gesture recognition system , 2001, Machine Vision and Applications.

[75]  Jian-Gang Wang,et al.  Eye gaze estimation from a single image of one eye , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[76]  Isaac Cohen,et al.  Classifying Facial Gestures in Presence of Head Motion , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[77]  Spyros G. Tzafestas,et al.  The autonomous mobile robot SENARIO: a sensor aided intelligent navigation system for powered wheelchairs , 1997, IEEE Robotics Autom. Mag..

[78]  J. L. Hodges,et al.  Discriminatory Analysis - Nonparametric Discrimination: Consistency Properties , 1989 .

[79]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[80]  John K. Tsotsos,et al.  Hand Gesture Recognition within a Linguistics-Based Framework , 2004, ECCV.

[81]  Zhan Yong-zhao,et al.  Facial expression recognition based on Gabor wavelet transformation and elastic templates matching , 2004 .

[82]  Marco La Cascia,et al.  Fully automatic, real-time detection of facial gestures from generic video , 2004, IEEE 6th Workshop on Multimedia Signal Processing, 2004..

[83]  M. Mazo,et al.  Guidance of a wheelchair for handicapped people by face tracking , 1999, 1999 7th IEEE International Conference on Emerging Technologies and Factory Automation. Proceedings ETFA '99 (Cat. No.99TH8467).

[84]  Oscar E. Agazzi,et al.  Machine vision for keyword spotting using pseudo 2D hidden Markov models , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[85]  Chung-Lin Huang,et al.  Hybrid-boost learning for multi-pose face detection and facial expression recognition , 2008, Pattern Recognit..

[86]  Yoshiaki Shirai,et al.  Interactive gesture interface for intelligent wheelchairs , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).