Transfer Learning Between Concepts for Human Behavior Modeling: An Application to Sincerity and Deception Prediction

Transfer learning (TL) involves leveraging information from sources outside the domain at hand for enhancing model performances. Popular TL methods either directly use the data or adapt the models learned on out-of-domain resources and incorporate them within in-domain models. TL methods have shown promise in several applications such as text classification, crossdomain language classification and emotion recognition. In this paper, we propose TL methods to computational human behavioral trait modeling. Many behavioral traits are abstract constructs (e.g., sincerity of an individual), and are often conceptually related to other constructs (e.g., level of deception) making TL methods an attractive option for their modeling. We consider the problem of automatically predicting human sincerity and deception from behavioral data while leveraging transfer of knowledge from each other. We compare our methods against baseline models trained only on in-domain data. Our best models achieve an Unweighted Average Recall (UAR) of 72.02% in classifying deception (baseline: 69.64%). Similarly, applied methods achieve Spearman’s/Pearson’s correlation values of 49.37%/48.52% between true and predicted sincerity scores (baseline: 46.51%/41.58%), indicating the success and the potential of TL for such human behavior tasks.

[1]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[2]  Yoshua Bengio,et al.  Deep Learning of Representations for Unsupervised and Transfer Learning , 2011, ICML Unsupervised and Transfer Learning.

[3]  Matthew S. Goodwin,et al.  Applying Machine Learning to Facilitate Autism Diagnostics: Pitfalls and Promises , 2015, Journal of autism and developmental disorders.

[4]  Y. V. Natochin,et al.  Integrity , 2019, Neuroscience and Behavioral Physiology.

[5]  Robert Herms,et al.  Prediction of Deception and Sincerity from Speech Using Automatic Phone Recognition-Based Features , 2016, INTERSPEECH.

[6]  B. Depaulo,et al.  Effects of actual deception and suspiciousness of deception on interpersonal perceptions. , 1984 .

[7]  K. Scherer,et al.  On the Acoustics of Emotion in Audio: What Speech, Music, and Sound have in Common , 2013, Front. Psychol..

[8]  Gábor Gosztolya,et al.  Estimating the Sincerity of Apologies in Speech by DNN Rank Learning and Prosodic Analysis , 2016, INTERSPEECH.

[9]  Yuan Shi,et al.  Geodesic flow kernel for unsupervised domain adaptation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Eduardo Coutinho,et al.  Enhanced semi-supervised learning for multimodal emotion recognition , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[11]  Wentong Cai,et al.  Agent‐based human behavior modeling for crowd simulation , 2008, Comput. Animat. Virtual Worlds.

[12]  Elmar Nöth,et al.  The INTERSPEECH 2015 computational paralinguistics challenge: nativeness, parkinson's & eating condition , 2015, INTERSPEECH.

[13]  Carla E. Brodley,et al.  Proceedings of the twenty-first international conference on Machine learning , 2004, International Conference on Machine Learning.

[14]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[15]  Heysem Kaya,et al.  Fusing Acoustic Feature Representations for Computational Paralinguistics Tasks , 2016, INTERSPEECH.

[16]  Nicu Sebe,et al.  We are not All Equal: Personalizing Models for Facial Expression Analysis with Transductive Parameter Transfer , 2014, ACM Multimedia.

[17]  Björn W. Schuller,et al.  Sincerity and Deception in Speech: Two Sides of the Same Coin? A Transfer- and Multi-Task Learning Perspective , 2016, INTERSPEECH.

[18]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[19]  Simona Caraiman,et al.  New applications of quantum algorithms to computer graphics: the quantum random sample consensus algorithm , 2009, CF '09.

[20]  John Blitzer,et al.  Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification , 2007, ACL.

[21]  Urbashi Mitra,et al.  2008 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) , 2008 .

[22]  Qiang Yang,et al.  Transfer Learning via Dimensionality Reduction , 2008, AAAI.

[23]  Rahul Gupta,et al.  Analysis of engagement behavior in children during dyadic interactions using prosodic cues , 2016, Comput. Speech Lang..

[24]  Eduardo Coutinho,et al.  The INTERSPEECH 2016 Computational Paralinguistics Challenge: Deception, Sincerity & Native Language , 2016, INTERSPEECH.

[25]  Jan-Michael Frahm,et al.  A Comparative Analysis of RANSAC Techniques Leading to Adaptive Real-Time Random Sample Consensus , 2008, ECCV.

[26]  Jan-Michael Frahm,et al.  USAC: A Universal Framework for Random Sample Consensus , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Alex Alves Freitas,et al.  On rule interestingness measures , 1999, Knowl. Based Syst..

[28]  James J. Jiang.  A Literature Survey on Domain Adaptation of Statistical Classifiers , 2007 .

[29]  John Blitzer,et al.  Domain Adaptation with Structural Correspondence Learning , 2006, EMNLP.

[30]  Yoshua Bengio,et al.  Unsupervised and Transfer Learning Challenge: a Deep Learning Approach , 2011, ICML Unsupervised and Transfer Learning.

[31]  Thomas G. Dietterich,et al.  Improving SVM accuracy by training on auxiliary data sources , 2004, ICML.

[32]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[33]  Peter H. Tu,et al.  Learning person-specific models for facial expression and action unit recognition , 2013, Pattern Recognit. Lett..

[34]  Lorien Y. Pratt,et al.  Discriminability-Based Transfer between Neural Networks , 1992, NIPS.

[35]  Bernt Schiele,et al.  Transfer Learning in a Transductive Setting , 2013, NIPS.

[36]  Ramesh Nallapati,et al.  A Comparative Study of Methods for Transductive Transfer Learning , 2007, Seventh IEEE International Conference on Data Mining Workshops (ICDMW 2007).

[37]  Peter Stone,et al.  Transfer Learning for Reinforcement Learning Domains: A Survey , 2009, J. Mach. Learn. Res..

[38]  Panayiotis G. Georgiou,et al.  Behavioral Signal Processing: Deriving Human Behavioral Informatics From Speech and Language , 2013, Proceedings of the IEEE.