Predicting Human Strategic Decisions Using Facial Expressions

People's facial expressions, whether made consciously or subconsciously, continuously reveal their state of mind. This work proposes a method for predicting people's strategic decisions based on their facial expressions. We designed a new version of the centipede game that intorduces an incentive for the human participant to hide her facial expressions. We recorded on video participants who played several games of our centipede version, and concurrently logged their decisions throughout the games. The video snippet of the participants' faces prior to their decisions is represented as a fixed-size vector by estimating the covariance matrix of key facial points which change over time. This vector serves as input to a classifier that is trained to predict the participant's decision. We compare several training techniques, all of which are designed to work with the imbalanced decisions typically made by the players of the game. Furthermore, we investigate adaptation of the trained model to each player individually, while taking into account the player's facial expressions in the previous games. The results show that our method outperforms standard SVM as well as humans in predicting subjects' strategic decisions. To the best of our knowledge, this is the first study to present a methodology for predicting people's strategic decisions when there is an incentive to hide facial expressions.

[1]  Yun Kyongsik,et al.  Emotional Interactions in Human Decision-Making using EEG Hyperscanning , 2008 .

[2]  K. Yun.  On the Same Wavelength: Face-to-Face Communication Increases Interpersonal Neural Synchronization , 2013, The Journal of Neuroscience.

[3]  N. Taatgen,et al.  What Eye Movements Can Tell about Theory of Mind in a Strategic Game , 2012, PloS one.

[4]  Jonathan Gratch,et al.  Bayesian model of the social effects of emotion in decision-making in multiagent systems , 2012, AAMAS.

[5]  Nello Cristianini,et al.  Controlling the Sensitivity of Support Vector Machines , 1999 .

[6]  J. Mycielski.  Games with perfect information , 1992 .

[7]  Samy Bengio,et al.  Discriminative keyword spotting , 2009, Speech Commun..

[8]  Ian Fasel,et al.  Inscrutable games? Facial expressions predict economic behavior , 2011, BMC Neuroscience.

[9]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[10]  R. Rosenthal.  Games of perfect information, predatory pricing and the chain-store paradox , 1981 .

[11]  Gwenn Englebienne,et al.  Move, and i will tell you who you are: detecting deceptive roles in low-quality data , 2011, ICMI '11.

[12]  A. Roth,et al.  Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria , 1998 .

[13]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[14]  Mehryar Mohri,et al.  AUC Optimization vs. Error Rate Minimization , 2003, NIPS.

[15]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[16]  Jonathan D. Cohen,et al.  The Neural Basis of Economic Decision-Making in the Ultimatum Game , 2003, Science.

[17]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[18]  Stephen Kwek,et al.  Applying Support Vector Machines to Imbalanced Datasets , 2004, ECML.