Real-Time Robot Manipulation Using Mouth Gestures in Facial Video Sequences

In this paper a novel method for the automatic command of three degrees of freedom of a robot using mouth gestures is presented. The method uses a set of different pixel-based segmentation algorithms and morphological restrictions in order to extract the mouth area from the frames. A fuzzy inference system is then used in order to produce a small subset of discriminante features for gesture classification. A state machine was designed in order to stabilize the robot command task by using a temporal sliding mean on the detected gestures. Experimental results show that the method is both robust and reliable when operated by different people, and fast enough to keep the detection's rate in real-time.

[1]  Alexander Zelinsky,et al.  Human-robot interaction using facial gesture recognition , 1996, Proceedings 5th IEEE International Workshop on Robot and Human Communication. RO-MAN'96 TSUKUBA.

[2]  Jean-Philippe Thiran,et al.  Automatic Extraction of Geometric Lip Features with Application to Multi-Modal Speaker Identification , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[3]  Tao Zhang,et al.  Gesture-based human-robot interaction using a knowledge-based software platform , 2006, Ind. Robot.

[4]  Patrice Delmas,et al.  Automatic lip tracking: Bayesian segmentation and active contours in a cooperative scheme , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[5]  Shu Hung Leung,et al.  Automatic lip contour extraction from color images , 2004, Pattern Recognit..

[6]  Anil K. Jain,et al.  Face Detection in Color Images , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  J. M. Sackier,et al.  Robotically assisted laparoscopic surgery , 2008, Surgical Endoscopy.

[8]  Alan Wee-Chung Liew,et al.  Lip segmentation with the presence of beards , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[9]  Rong Chen,et al.  A PCA Based Visual DCT Feature Extraction Method for Lip-Reading , 2006, 2006 International Conference on Intelligent Information Hiding and Multimedia.

[10]  Tom E. Bishop,et al.  Blind Image Restoration Using a Block-Stationary Signal Model , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[11]  Juan Jesús Fernández Lozano,et al.  A medical robotic assistant for minimally invasive surgery , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[12]  Alice Caplier,et al.  New color transformation for lips segmentation , 2001, 2001 IEEE Fourth Workshop on Multimedia Signal Processing (Cat. No.01TH8564).

[13]  Trent W. Lewis,et al.  Lip Feature Extraction Using Red Exclusion , 2000, VIP.

[14]  Tanneguy Redarce,et al.  Lips Movement Segmentation and Features Extraction in Real Time , 2007 .

[15]  Fumio Miyazaki,et al.  FAce MOUSe: A novel human-machine interface for controlling the position of a laparoscope , 2003, IEEE Trans. Robotics Autom..

[16]  M. Eckert,et al.  Compensación de movimiento avanzada para codificación de vídeo , 2002 .