Real-time system for measuring gaze direction and facial features: towards automatic discrimination of lies using diverse nonverbal information

Interactive and autonomous agents might be common in everyday life in the future; we expect that such agents will have the ability to communicate with people naturally. For natural communication, the agents should speculate about the intentions of the people they interact with. To enable agents to speculate about intentions like deception, we focused on unconscious expressions when people tell a lie. However, there is no system that can meet the necessary conditions for measuring nonverbal information in natural communication. Therefore, we made a real-time system for measuring gaze direction and facial features. We conducted experiments for discriminating lies by using the system in a situation similar to actual communication. As a result, we found that we could discriminate lies by using diverse nonverbal information in the same way people did.

[1]  Walburga Von Raffler-Engel,et al.  Aspects of nonverbal communication , 1980 .

[2]  K. Fukuda,et al.  Eye blinks: new indices for the detection of deception. , 2001, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[3]  James J. Lindsay,et al.  Cues to deception. , 2003, Psychological bulletin.

[4]  Shinjiro Kawato,et al.  Detection and tracking of eyes for gaze-camera control , 2004, Image Vis. Comput..

[5]  Paul Kay,et al.  Prototype semantics: The English Word Lie , 1981 .

[6]  Naoki Mukawa,et al.  FreeGaze: a gaze tracking system for everyday gaze interaction , 2002, ETRA.

[7]  S Porchet-Munro,et al.  Aspects of nonverbal communication. , 1991, Recent results in cancer research. Fortschritte der Krebsforschung. Progres dans les recherches sur le cancer.

[8]  電子情報通信学会.  The Transactions of the Institute of Electronics, Information and Communication Engineers , 1987 .

[9]  Shinjiro Kawato,et al.  Scale-Adaptive Face Detection and Tracking in Real Time with SSR Filters and Support Vector Machine , 2005, IEICE Trans. Inf. Syst..

[10]  大坊 郁夫,et al.  The social meaning of interpersonal communication , 2001 .

[11]  Shinjiro Kawato,et al.  A nose-tracked, personal olfactory display , 2003, SIGGRAPH '03.

[12]  Yoichi Sato,et al.  Head Pose Estimation System Based on Particle Filtering with Adaptive Diffusion Control , 2005, MVA.

[13]  G. Miller,et al.  Handbook of Interpersonal Communication , 1985 .

[14]  Alexander Zelinsky,et al.  An algorithm for real-time stereo vision implementation of head pose and gaze direction measurement , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).