A Peak Detection Method for Understanding User States for Empathetic Intelligent Agents

Recognition of facial expression is a useful and unobtrusive means for machines to understand users' internal states. However, most human facial expression is ambiguous or subtle to recognize resulting in poor accuracy. To overcome this limitation, we propose a peak detection method to select only meaningful images from image sequences which imply significant changes of facial expression by calculating differences between images. We believe this method will alleviate the accuracy degradation caused by different personal appearances and ambiguous facial expressions. When applied to commercial products, it can provides a suitable method for adaptation of the empathetic agent embedded in machines such as personal assistants on TV, smartphone and vehicles based on recognized facial expression of users. For experimental validation of the proposed method, we tested four different features for measuring image similarity with the extended Cohn-Kanade facial image dataset. As a result, we could get better recognition accuracy than the original image sequences. Moreover, we reduced the number of images that need to be recognized to 24.52% without degradation of accuracy.

[1]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[2]  Ian D. Reid,et al.  Stable multi-target tracking in real-time surveillance video , 2011, CVPR 2011.

[3]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[4]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[5]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[6]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[8]  Yang Yu,et al.  Automatic image annotation using group sparsity , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[10]  Shuicheng Yan,et al.  An HOG-LBP human detector with partial occlusion handling , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[11]  Matti Pietikäinen,et al.  Performance evaluation of texture measures with classification based on Kullback discrimination of distributions , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[12]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[13]  Fumio Hara,et al.  Recognition of Six basic facial expression and their strength by neural network , 1992, [1992] Proceedings IEEE International Workshop on Robot and Human Communication.

[14]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[15]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[16]  Shaogang Gong,et al.  Robust facial expression recognition using local binary patterns , 2005, IEEE International Conference on Image Processing 2005.

[17]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[18]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[19]  Thomas S. Huang,et al.  Emotion Recognition from Facial Expressions using Multilevel HMM , 2000 .