Autonomous Robot Control Using Facial Expressions

In this paper, a novel real time robot control system using human facial expressions is presented. The proposed system mainly consists of three modules: face detection, facial expression recognition and robot control. The first module aims to find the user's face from the image captured from a live video through a series of steps including skin color classification, edge detection and mathematical morphology, while the second module analyses the detected face to recognize the facial expressions like happiness, sadness, surprise, anger and neutral using Principal Component Analysis (PCA) and Euclidian distance calculation. Finally, the detected facial expressions are used as controlling commands for the robot. Experiments were conducted for respective modules. 150 images under different lightening conditions and complex background were employed. In the experiment on detecting the facial region, the rate of detection was 99.3% and on recognizing the facial expression, the recognition rate was 97.3%. The robot was then controlled autonomously with an accuracy of 100%. Combining the above three rates, the overall rate of success of our system was 97.3%.

[1]  Takeo Kanade,et al.  Recognizing Action Units for Facial Expression Analysis , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[4]  T. Lalanne,et al.  Color recognition with a camera: a supervised algorithm for classification , 1998, 1998 IEEE Southwest Symposium on Image Analysis and Interpretation (Cat. No.98EX165).

[5]  Rodrigo A. M. Braga,et al.  Interface Framework to Drive an Intelligent Wheelchair Using Facial Expressions , 2007, 2007 IEEE International Symposium on Industrial Electronics.

[6]  Yun He,et al.  Automatic face segmentation using color cues for coding typical videophone scenes , 1997, Electronic Imaging.

[7]  King Ngi Ngan,et al.  Face segmentation using skin-color map in videophone applications , 1999, IEEE Trans. Circuits Syst. Video Technol..

[8]  Yuxin Wang,et al.  A New Approach for Color-Based Object Recognition with Fusion of Color Models , 2008, 2008 Congress on Image and Signal Processing.

[9]  Richard A. Foulds,et al.  Toward robust skin identification in video images , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[10]  Eun Yi Kim,et al.  Intelligent wheelchair (IW) interface using face and mouth recognition , 2009, IUI.

[11]  King Ngi Ngan,et al.  Locating facial region of a head-and-shoulders color image , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[12]  Anil K. Jain,et al.  Face Detection in Color Images , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  John R. Kender,et al.  Finding skin in color images , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[14]  Erik Hjelmås,et al.  Face Detection: A Survey , 2001, Comput. Vis. Image Underst..

[15]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[17]  Ioannis Pitas,et al.  Face localization and facial feature extraction based on shape and color information , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[18]  Jong-Baem Won,et al.  An intelligent control of mobile robot based on voice command , 2010, ICCAS 2010.

[19]  King Ngi Ngan,et al.  Automatic face location for videophone images , 1996, Proceedings of Digital Processing Applications (TENCON '96).