Generalization of a Vision-Based Computational Model of Mind-Reading

This paper describes a vision-based computational model of mind-reading that infers complex mental states from head and facial expressions in real-time. The generalization ability of the system is evaluated on videos that were posed by lay people in a relatively uncontrolled recording environment for six mental states—agreeing, concentrating, disagreeing, interested, thinking and unsure. The results show that the system’s accuracy is comparable to that of humans on the same corpus.

[1]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[2]  el Kaliouby,et al.  Mind-reading machines: automated inference of complex mental states , 2005 .

[3]  Wade Junek,et al.  Mind Reading: The Interactive Guide to Emotions , 2007 .

[4]  P. Ekman.  Pictures of Facial Affect , 1976 .

[5]  Gwen Littlewort,et al.  Dynamics of Facial Expression Extracted Automatically from Video , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[6]  Yuri Ivanov,et al.  Probabilistic combination of multiple modalities to detect interest , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[7]  Qiang Ji,et al.  Facial event classification with task oriented dynamic Bayesian network , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[8]  D. Povinelli,et al.  Mindblindness. An Essay on Autism and Theory of Mind Simon Baron-Cohen 1995 , 1996, Trends in Neurosciences.

[9]  Peter Robinson,et al.  Real-Time Inference of Complex Mental States from Facial Expressions and Head Gestures , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[10]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[11]  Lisa M. Brown,et al.  Real World Real-time Automatic Recognition of Facial Expressions , 2003 .

[12]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  S. Baron-Cohen,et al.  The "Reading the Mind in the Eyes" Test revised version: a study with normal adults, and adults with Asperger syndrome or high-functioning autism. , 2001, Journal of child psychology and psychiatry, and allied disciplines.

[14]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[15]  Montse Pardàs,et al.  Emotion recognition based on MPEG-4 Facial Animation Parameters , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[16]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .