A Multimodal Dataset for Deception Detection

This paper presents the construction of a multimodal dataset for deception detection, including physiological, thermal, and visual responses of human subjects under three deceptive scenarios. We present the experimental protocol, as well as the data acquisition process. To evaluate the usefulness of the dataset for the task of deception detection, we present a statistical analysis of the physiological and thermal modalities associated with the deceptive and truthful conditions. Initial results show that physiological and thermal responses can differentiate between deceptive and truthful states.

[1]  David M. Howard,et al.  Acoustic Correlates of Deceptive Speech - An Exploratory Study , 2011, HCI.

[2]  Jay F. Nunamaker,et al.  Modality Effects in Deception Detection and Applications in Automatic-Deception-Detection , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[3]  A. Freitas-Magalhães.  Facial Expression of Emotion , 2012 .

[4]  James Edwin Mahon,et al.  A Definition of Deceiving , 2007 .

[5]  S. Porter,et al.  Pitfalls and Opportunities in Nonverbal and Verbal Lie Detection , 2010, Psychological science in the public interest : a journal of the American Psychological Society.

[6]  Björn W. Schuller,et al.  OpenEAR — Introducing the munich open-source emotion and affect recognition toolkit , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[7]  Bruno Verschuere,et al.  Detecting Criminal Intent with the Concealed Information Test , 2010 .

[8]  Andreas Stolcke,et al.  Distinguishing deceptive from non-deceptive speech , 2005, INTERSPEECH.

[9]  I. Pavlidis,et al.  Forehead Thermal Signature Extraction in Lie Detection , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[10]  Gwen Littlewort,et al.  The computer expression recognition toolbox (CERT) , 2011, Face and Gesture 2011.

[11]  Peng Hao,et al.  Adaptive Context Modeling for Deception Detection in Emails , 2011, MLDM.

[12]  J. Pennebaker,et al.  Lying Words: Predicting Deception from Linguistic Styles , 2003, Personality & social psychology bulletin.

[13]  Jennifer S. Beer,et al.  Facial expression of emotion. , 2003 .

[14]  S. Porter,et al.  Secrets and Lies: Involuntary Leakage in Deceptive Facial Expressions as a Function of Emotional Intensity , 2011, Journal of Nonverbal Behavior.