How Human Am I?: EEG-based Evaluation of Virtual Characters

There is a continuous effort by animation experts to create increasingly realistic and more human-like digital characters. However, as virtual characters become more human they risk evoking a sense of unease in their audience. This sensation, called the Uncanny Valley effect, is widely acknowledged both in the popular media and scientific research but empirical evidence for the hypothesis has remained inconsistent. In this paper, we investigate the neural responses to computer-generated faces in a cognitive neuroscience study. We record brain activity from participants (N = 40)} using electroencephalography (EEG) while they watch videos of real humans and computer-generated virtual characters. Our results show distinct differences in neural responses for highly realistic computer-generated faces such as Digital Emily compared with real humans. These differences are unique only to agents that are highly photorealistic, i.e. the `uncanny' response. Based on these specific neural correlates we train a support vector machine~(SVM) to measure the probability of an uncanny response for any given computer-generated character from EEG data. This allows the ordering of animated characters based on their level of `uncanniness'.

[1]  Michèle Fabre-Thorpe,et al.  Animal and human faces in natural scenes: How specific to human faces is the N170 ERP component? , 2004, Journal of vision.

[2]  Margot J. Taylor,et al.  N170 or N1? Spatiotemporal differences between object and face processing using ERPs. , 2004, Cerebral cortex.

[3]  Guy L. Lacroix,et al.  Does the uncanny valley exist? An empirical test of the relationship between eeriness and the human likeness of digitally created faces , 2013, Comput. Hum. Behav..

[4]  H. Ishiguro,et al.  The uncanny advantage of using androids in cognitive and social science research , 2006 .

[5]  Kara D. Federmeier,et al.  Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). , 2011, Annual review of psychology.

[6]  K. MacDorman,et al.  Subjective Ratings of Robot Video Clips for Human Likeness, Familiarity, and Eeriness: An Exploration of the Uncanny Valley , 2006 .

[7]  Arjun Nagendran,et al.  Expectancy Violations Related to a Virtual Human ’ s Joint Gaze Behavior in Real-Virtual Human Interactions , .

[8]  H. Neville,et al.  Natural speech processing: An analysis using event-related brain potentials , 1991, Psychobiology.

[9]  H. Ishiguro,et al.  The thing that should not be: predictive coding and the uncanny valley in perceiving human and humanoid robot actions , 2011, Social cognitive and affective neuroscience.

[10]  Karl F. MacDorman,et al.  Too real for comfort? Uncanny responses to computer generated faces , 2009, Comput. Hum. Behav..

[11]  Thomas Serre,et al.  The Neural Dynamics of Face Detection in the Wild Revealed by MVPA , 2014, The Journal of Neuroscience.

[12]  Mark Grimshaw,et al.  Uncanny behaviour in survival horror games , 2010 .

[13]  Marcus Cheetham,et al.  Category Processing and the human likeness dimension of the Uncanny Valley Hypothesis: Eye-Tracking Data , 2013, Front. Psychol..

[14]  Tim Weyrich,et al.  Analysis of human faces using a measurement-based skin reflectance model , 2006, ACM Trans. Graph..

[15]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[16]  Nicole C. Krämer,et al.  How design characteristics of robots determine evaluation and uncanny valley related responses , 2014, Comput. Hum. Behav..

[17]  Diego Gutierrez,et al.  To stylize or not to stylize? , 2015, ACM Trans. Graph..

[18]  Andrew Jones,et al.  Digital Ira: creating a real-time photoreal digital actor , 2013, SIGGRAPH '13.

[19]  Nigam,et al.  N400 to Semantically Anomalous Pictures and Words , 1992, Journal of Cognitive Neuroscience.

[20]  Martin Breidt,et al.  Face reality: investigating the Uncanny Valley for virtual faces , 2010, SIGGRAPH ASIA.

[21]  Matthias Scheutz,et al.  Too Much Humanness for Human-Robot Interaction: Exposure to Highly Humanlike Robots Elicits Aversive Responding in Observers , 2015, CHI.

[22]  Alessandra Sciutti,et al.  Cognition: A Bridge between Robotics and Interaction , 2015, HRI.

[23]  T. Allison,et al.  Electrophysiological Studies of Face Perception in Humans , 1996, Journal of Cognitive Neuroscience.

[24]  D. Jeffreys Evoked Potential Studies of Face and Object Processing , 1996 .

[25]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[26]  Mark Grimshaw,et al.  Facial expression of emotion and perception of the Uncanny Valley in virtual characters , 2011, Comput. Hum. Behav..

[27]  Alison Oddey,et al.  Modes of Spectating , 2009 .

[28]  Benjamin Balas,et al.  EEG correlates of categorical and graded face perception , 2011, Neuropsychologia.

[29]  M. Gross,et al.  Analysis of human faces using a measurement-based skin reflectance model , 2006, ACM Trans. Graph..

[30]  Valentin Schwind,et al.  The Uncanny Valley and the Importance of Eye Contact , 2016, i-com.

[31]  Heloir,et al.  The Uncanny Valley , 2019, The Animation Studies Reader.

[32]  L. Jäncke,et al.  Human Neuroscience , 2022 .

[33]  J. Davidoff,et al.  Brain events related to normal and moderately scrambled faces. , 1996, Brain research. Cognitive brain research.

[34]  K. Bötzel,et al.  Scalp topography and analysis of intracranial sources of face-evoked potentials , 2004, Experimental Brain Research.

[35]  D. Friedman,et al.  The novelty P3: an event-related brain potential (ERP) sign of the brain's evaluation of novelty , 2001, Neuroscience & Biobehavioral Reviews.

[36]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[37]  Edward Cutrell,et al.  Measuring the Perception of Visual Realism in Images , 2001, Rendering Techniques.

[38]  Tom Geller,et al.  Overcoming the Uncanny Valley , 2008, IEEE Computer Graphics and Applications.

[39]  Wan-Chun Ma,et al.  The Digital Emily Project: Achieving a Photorealistic Digital Actor , 2010, IEEE Computer Graphics and Applications.

[40]  Jun'ichiro Seyama,et al.  The Uncanny Valley: Effect of Realism on the Impression of Artificial Human Faces , 2007, PRESENCE: Teleoperators and Virtual Environments.

[41]  Karl F. MacDorman,et al.  Human emotion and the uncanny valley: A GLM, MDS, and Isomap analysis of robot video ratings , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[42]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[43]  Anton Leuski,et al.  All Together Now - Introducing the Virtual Human Toolkit , 2013, IVA.

[44]  Marcus A. Magnor,et al.  EEG Based Analysis of the Perception of Computer-Generated Faces , 2016, CVMP 2016.