Perception of virtual characters

This course will introduce students, researchers and digital artists to the recent results in perceptual research on virtual characters. It covers both how technical and artistic aspects that constitute the appearance of a virtual character influence human perception. We will report results of studies that addressed the influence of low-level cues like facial proportions, shading or level of detail and higher-level cues such as behavior or artistic stylization. We will place emphasis on aspects that are encountered during character development, animation, and achieving consistency between the visuals and storytelling. The insights that we present in this course will serve as an additional toolset to anticipate the effect of certain design decisions and to create more convincing characters, especially in the case where budgets or time are limited.

[1]  Norman I. Badler,et al.  A Review of Eye Gaze in Virtual Agents, Social Robotics and HCI: Behaviour Generation, User Interaction and Perception , 2015, Comput. Graph. Forum.

[2]  David H Brainard,et al.  Joint effects of illumination geometry and object shape in the perception of surface reflectance , 2011, i-Perception.

[3]  Ken Perlin,et al.  Realistic human body movement for emotional expressiveness , 2009, SIGGRAPH '09.

[4]  T. K. Srull,et al.  A Dual process model of impression formation , 1988 .

[5]  D. Samuel Schwarzkopf,et al.  The surface area of human V1 predicts the subjective experience of object size , 2010, Nature Neuroscience.

[6]  Jonas Beskow,et al.  Reverse Engineering Psychologically Valid Facial Expressions of Emotion into Social Robots , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[7]  Jeremy N. Bailenson,et al.  A meta-analysis of the impact of the inclusion and realism of human-like faces on user experiences in interfaces , 2007, CHI.

[8]  Gary Bishop,et al.  Measuring the perceived visual realism of images , 2003 .

[9]  J. Rauthmann,et al.  Eyes as windows to the soul: Gazing behavior is related to personality , 2012 .

[10]  Edward Cutrell,et al.  Measuring the Perception of Visual Realism in Images , 2001, Rendering Techniques.

[11]  P. Borkenau,et al.  Trait inferences: Sources of validity at zero acquaintance. , 1992 .

[12]  A. Greenwald,et al.  Measuring individual differences in implicit cognition: the implicit association test. , 1998, Journal of personality and social psychology.

[13]  M. Landy,et al.  How direction of illumination affects visually perceived surface roughness. , 2006, Journal of vision.

[14]  N. Frijda.  The laws of emotion. , 1988, The American psychologist.

[15]  Rachel McDonnell,et al.  Does render style affect perception of personality in virtual humans? , 2014, SAP.

[16]  A. Mehrabian.  Basic dimensions for a general psychological theory : implications for personality, social, environmental, and developmental studies , 1980 .

[17]  Fabio Pianesi,et al.  A first evaluation study of a database of kinetic facial expressions (DaFEx) , 2005, ICMI '05.

[18]  Daniel Thalmann,et al.  EG 2007 Course on Populating Virtual Environments with Crowds , 2007, Eurographics.

[19]  Heinrich H. Bülthoff,et al.  Evaluating the perceptual realism of animated facial expressions , 2008, TAP.

[20]  Frank E. Pollick,et al.  He throws like a girl (but only when he’s sad): Emotion affects sex-decoding of biological motion displays , 2011, Cognition.

[21]  Rachel McDonnell,et al.  Perceptually Adaptive Graphics , 2004, Eurographics.

[22]  Jun'ichiro Seyama,et al.  The Uncanny Valley: Effect of Realism on the Impression of Artificial Human Faces , 2007, PRESENCE: Teleoperators and Virtual Environments.

[23]  Michael J. Black,et al.  Appealing Female Avatars from 3D Body Scans: Perceptual Effects of Stylization , 2016, VISIGRAPP.

[24]  Jessica K. Hodgins,et al.  Perceptual effects of damped and exaggerated facial motion in animated characters , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[25]  Roland W Fleming,et al.  Real-world illumination and the perception of surface reflectance properties. , 2003, Journal of vision.

[26]  Soraia Raupp Musse,et al.  Evaluation of the Uncanny Valley in CG Characters , 2012, IVA.

[27]  A. Young,et al.  Emotion Perception from Dynamic and Static Body Expressions in Point-Light and Full-Light Displays , 2004, Perception.

[28]  Ludwig. Kardos,et al.  Ding und Schatten : eine experimentelle untersuchung über die Grundlagen des Farbensehens , 1934 .

[29]  Rachel McDonnell,et al.  Perceptual Evaluation of Impostor Representations for Virtual Humans and Buildings , 2005, Comput. Graph. Forum.

[30]  G. Johansson.  Visual perception of biological motion and a model for its analysis , 1973 .

[31]  Heinrich H. Bülthoff,et al.  Low-Level Image Cues in the Perception of Translucent Materials , 2005, TAP.

[32]  Xiao Yang,et al.  Modeling style and variation in human motion , 2010, SCA 2010.

[33]  Ludovic Hoyet,et al.  Exploring the Effect of Motion Type and Emotions on the Perception of Gender in Virtual Humans , 2015, ACM Trans. Appl. Percept..

[34]  M. D. Meijer.  The contribution of general features of body movement to the attribution of emotions , 1989 .

[35]  Marc Erich Latoschik,et al.  The Impact of Avatar Personalization and Immersion on Virtual Body Ownership, Presence, and Emotional Response , 2018, IEEE Transactions on Visualization and Computer Graphics.

[36]  J. Cutting,et al.  Recognizing the sex of a walker from a dynamic point-light display , 1977 .

[37]  Erik Geslin.  Method of induction of basic and complex emotions in video games and virtual environments , 2012, SA '12.

[38]  Libor Vása,et al.  Perceptual Metrics for Static and Dynamic Triangle Meshes , 2013, Comput. Graph. Forum.

[39]  M. Cunningham,et al.  Article Commentary: Averaged Faces Are Attractive, but Very Attractive Faces Are Not Average , 1991 .

[40]  Daniel Thalmann,et al.  Populating virtual environments with crowds , 2006, VRCIA '06.

[41]  D. Brainard.  5 – Color Appearance and Color Difference Specification , 2003 .

[42]  P. Hancock,et al.  The role of masculinity and distinctiveness in judgments of human male facial attractiveness. , 2002, British journal of psychology.

[43]  R. Plutchik.  Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice , 2016 .

[44]  Rachel McDonnell,et al.  Perception of personality through eye gaze of realistic and cartoon models , 2015, SAP.

[45]  D. C. Beardslee,et al.  Readings in perception , 1958 .

[46]  R. Groner,et al.  Looking at Faces: Local and Global Aspects of Scanpaths , 1984 .

[47]  B. Fink,et al.  The effects of skin colour distribution and topography cues on the perception of female facial age and health , 2008, Journal of the European Academy of Dermatology and Venereology : JEADV.

[48]  Jean-Claude Martin,et al.  Combining Facial and Postural Expressions of Emotions in a Virtual Character , 2009, IVA.

[49]  Nuria Pelechano,et al.  Simulating Heterogeneous Crowds with Interactive Behaviors , 2014, Eurographics.

[50]  John Dingliana,et al.  Perception of lighting and shading for animated virtual characters , 2016, SAP.

[51]  J. Hodgins,et al.  Anthropomorphism influences perception of computer-animated characters' actions. , 2007, Social cognitive and affective neuroscience.

[52]  Heinrich H. Bülthoff,et al.  The components of conversational facial expressions , 2004, APGV '04.

[53]  Armin Bruderlin,et al.  Perceiving affect from arm movement , 2001, Cognition.

[54]  A. Tellegen,et al.  PERSONALITY PROCESSES AND INDIVIDUAL DIFFERENCES An Alternative "Description of Personality": The Big-Five Factor Structure , 2022 .

[55]  Antony S R Manstead,et al.  Gender and culture differences in emotion. , 2004, Emotion.

[56]  Valentin Schwind,et al.  The Uncanny Valley and the Importance of Eye Contact , 2016, i-com.

[57]  Daniel Thalmann,et al.  Course: Modeling Individualities in Groups and Crowds , 2009, Eurographics.

[58]  K. Grammer,et al.  Skin color distribution plays a role in the perception of age, attractiveness and health in female faces , 2006 .

[59]  K. Bala,et al.  Effects of global illumination approximations on material appearance , 2010, SIGGRAPH 2010.

[60]  Heinrich H. Bülthoff,et al.  Render me real? , 2012, ACM Trans. Graph..

[61]  Sun Joo Grace Ahn,et al.  Self-Endorsing Versus Other-Endorsing in Virtual Environments , 2011 .

[62]  Dirk Bartz,et al.  The Role of Perception for Computer Graphics , 2008, Eurographics.

[63]  Eakta Jain,et al.  Identifying Computer-Generated Portraits: an Eye Tracking Study , 2018 .

[64]  Diego Gutierrez,et al.  Exploiting perception in high-fidelity virtual environments (Additional presentations from the 24th course are available on the citation page) , 2006, SIGGRAPH Courses.

[65]  Michael Neff,et al.  Evaluating the Effect of Gesture and Language on Personality Perception in Conversational Agents , 2010, IVA.

[66]  Simon Richir,et al.  From Robot to Virtual Doppelganger: Impact of Visual Fidelity of Avatars Controlled in Third-Person Perspective on Embodiment and Behavior in Immersive Virtual Environments , 2019, Front. Robot. AI.

[67]  Alexander Keller,et al.  Instant radiosity , 1997, SIGGRAPH.

[68]  Karl F. MacDorman,et al.  Too real for comfort? Uncanny responses to computer generated faces , 2009, Comput. Hum. Behav..

[69]  Kathryn Y. Segovia,et al.  Virtual imposters: Responses to avatars that do not look like their controllers , 2012 .

[70]  G. Rhodes.  The evolutionary psychology of facial beauty. , 2006, Annual review of psychology.

[71]  Elizabeth A. Crane,et al.  Motion Capture and Emotion: Affect Detection in Whole Body Movement , 2007, ACII.

[72]  T. Wheatley,et al.  The Tipping Point of Animacy , 2010, Psychological science.

[73]  K. MacDorman,et al.  Subjective Ratings of Robot Video Clips for Human Likeness, Familiarity, and Eeriness: An Exploration of the Uncanny Valley , 2006 .

[74]  Laurence T Maloney,et al.  The proximity structure of achromatic surface colors and the impossibility of asymmetric lightness matching , 2006, Perception & psychophysics.

[75]  Diego Gutierrez,et al.  To stylize or not to stylize? , 2015, ACM Trans. Graph..

[76]  Sabarish V. Babu,et al.  Effects of Virtual Human Appearance Fidelity on Emotion Contagion in Affective Inter-Personal Simulations , 2016, IEEE Transactions on Visualization and Computer Graphics.

[77]  D. Marlowe,et al.  A new scale of social desirability independent of psychopathology. , 1960, Journal of consulting psychology.

[78]  Carol O'Sullivan,et al.  Perceiving human motion variety , 2011, APGV '11.

[79]  M. Slater,et al.  Illusory ownership of a virtual child body causes overestimation of object sizes and implicit attitude changes , 2013, Proceedings of the National Academy of Sciences.

[80]  Carol O'Sullivan,et al.  Perception of simplification artifacts for animated characters , 2011, APGV '11.

[81]  N. Campbell,et al.  Facial movement varies by sex and is related to attractiveness , 2007 .

[82]  Jessica K. Hodgins,et al.  Interactive control of avatars animated with human motion data , 2002, SIGGRAPH.

[83]  P. Ekman.  An argument for basic emotions , 1992 .

[84]  J. Russell.  A circumplex model of affect. , 1980 .

[85]  Maria Olkkonen,et al.  Perceived glossiness and lightness under real-world illumination. , 2010, Journal of vision.

[86]  A. Johnston,et al.  Categorizing sex and identity from the biological motion of faces , 2001, Current Biology.

[87]  A. Jensen,et al.  Scoring the Stroop test. , 1965, Acta psychologica.

[88]  Susan Persky,et al.  The Independent and Interactive Effects of Embodied-Agent Appearance and Behavior on Self-Report, Cognitive, and Behavioral Markers of Copresence in Immersive Virtual Environments , 2005, Presence: Teleoperators & Virtual Environments.

[89]  Wendy Ju,et al.  Social robots and virtual agents as lecturers for video instruction , 2016, Comput. Hum. Behav..

[90]  Carol O'Sullivan,et al.  Perceptual evaluation of LOD clothing for virtual humans , 2006, SCA '06.

[91]  Hany Farid,et al.  Identifying Computer-Generated Portraits: The Importance of Training and Incentives , 2017, Perception.

[92]  Michael J. Black,et al.  Visual Perception and Evaluation of Photo-Realistic Self-Avatars From 3D Body Scans in Males and Females , 2018, Front. ICT.

[93]  B. Nitzsche,et al.  Bodyshop: The Photoshop Retouching Guide for the Face and Body , 2010 .

[94]  W. Köhler.  A Brief Introduction to Gestalt, Identifying Key Theories and Principles , 2007 .

[95]  Roger J. Hubbold,et al.  Perceptually‐Driven Radiosity , 1997, Comput. Graph. Forum.

[96]  P. Niedenthal,et al.  Emotional state and the detection of change in facial expression of emotion , 2000 .

[97]  T. Denson,et al.  Evidence from Meta-Analyses of the Facial Width-to-Height Ratio as an Evolved Cue of Threat , 2015, PloS one.

[98]  R. Plutchik.  The Nature of Emotions , 2001 .

[99]  S. Gosling,et al.  Personality in its natural habitat: manifestations and implicit folk theories of personality in daily life. , 2006, Journal of personality and social psychology.

[100]  P. Costa,et al.  NEO inventories for the NEO Personality Inventory-3 (NEO-PI-3), NEO Five-Factor Inventory-3 (NEO-FFI-3), NEO Personality Inventory-Revised (NEO PI-R) : professional manual , 2010 .

[101]  Guy L. Lacroix,et al.  Does the uncanny valley exist? An empirical test of the relationship between eeriness and the human likeness of digitally created faces , 2013, Comput. Hum. Behav..

[102]  Karl F. MacDorman,et al.  Revisiting the uncanny valley theory: Developing and validating an alternative to the Godspeed indices , 2010, Comput. Hum. Behav..

[103]  Daniel Thalmann,et al.  Believable Virtual Characters in Human-Computer Dialogs , 2011, Eurographics.

[104]  Yifan Wang,et al.  Exploring the Uncanny Valley with Japanese Video Game Characters , 2007, DiGRA Conference.

[105]  M. Wertheimer.  Untersuchungen zur Lehre von der Gestalt. II , 1923 .

[106]  Peter Vangorp,et al.  Human Visual Perception of Materials in Realistic Computer Graphics , 2009 .

[107]  Heinrich H. Bülthoff,et al.  Evaluation of real-world and computer-generated stylized facial expressions , 2007, TAP.

[108]  Jessica K. Hodgins,et al.  Evaluating the effect of motion and body shape on the perceived sex of virtual characters , 2009, TAP.

[109]  Philip Dutré,et al.  The influence of shape on the perception of material reflectance , 2007, ACM Trans. Graph..

[110]  Carol O'Sullivan,et al.  Clone attack! Perception of crowd variety , 2008, SIGGRAPH 2008.

[111]  Rachel McDonnell,et al.  The Effect of Realistic Appearance of Virtual Characters in Immersive Environments - Does the Character's Personality Play a Role? , 2018, IEEE Transactions on Visualization and Computer Graphics.

[112]  Erik Reinhard,et al.  Perceptually-Motivated Graphics , 2008, Eurographics.

[113]  Carol O'Sullivan,et al.  Eye-catching crowds: saliency based selective variation , 2009, SIGGRAPH 2009.

[114]  Veronica Sundstedt,et al.  Crowd Light: Evaluating the Perceived Fidelity of Illuminated Dynamic Scenes , 2012, Comput. Graph. Forum.

[115]  Karl F. MacDorman,et al.  Sensitivity to the proportions of faces that vary in human likeness , 2008, Comput. Hum. Behav..

[116]  Ming C. Lin,et al.  Simulating believable crowd and group behaviors , 2010, SA '10.

[117]  Kerri L. Johnson,et al.  Compatibility of basic social perceptions determines perceived attractiveness , 2007, Proceedings of the National Academy of Sciences.

[118]  Manish Singh,et al.  Visual perception of 3D shape , 2009, SIGGRAPH '09.

[119]  Ludovic Hoyet,et al.  Emotion Capture: Emotionally Expressive Characters for Games , 2013, MIG.

[120]  L. Zebrowitz,et al.  "Wide-Eyed" and "Crooked-Faced": Determinants of Perceived and Real Honesty Across the Life Span , 1996 .

[121]  Carol O'Sullivan,et al.  Movements and voices affect perceived sex of virtual conversers , 2010, APGV '10.

[122]  Sophie Jörg,et al.  Investigating the role of body shape on the perception of emotion , 2009, TAP.

[123]  Marina Daecher,et al.  Level Of Detail For 3d Graphics , 2016 .

[124]  Karl F. MacDorman,et al.  The doctor’s digital double: how warmth, competence, and animation promote adherence intention , 2018, PeerJ Comput. Sci..

[125]  Ekaterina P. Volkova,et al.  Emotion categorization of body expressions in narrative scenarios , 2014, Front. Psychol..

[126]  Jennifer Lee Poland.  LIGHTS, CAMERA, EMOTION! AN EXAMINATION ON FILM LIGHTING AND ITS IMPACT ON AUDIENCES’ EMOTIONAL RESPONSE , 2015 .

[127]  Marie L. Smith,et al.  Transmission of Facial Expressions of Emotion Co-Evolved with Their Efficient Decoding in the Brain: Behavioral and Brain Evidence , 2009, PloS one.

[128]  H. Blackwell,et al.  Studies of psychophysical methods for measuring visual thresholds. , 1952, Journal of the Optical Society of America.

[129]  Andrew Herbert,et al.  Reading Personality: Avatar vs. Human Faces , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[130]  J. Bailenson,et al.  Virtual Self-Modeling: The Effects of Vicarious Reinforcement and Identification on Exercise Behaviors , 2009 .

[131]  D. Keltner,et al.  The Gender Stereotyping of Emotions , 2000 .

[132]  S. Pont,et al.  Material — Illumination Ambiguities and the Perception of Solid Objects , 2006, Perception.

[133]  Jessica K. Hodgins,et al.  The saliency of anomalies in animated human characters , 2010, TAP.

[134]  Daniel Thalmann,et al.  EG 2006 Course on Populating Virtual Environments with Crowds , 2006, Eurographics.

[135]  Connor J. Parde,et al.  First Impressions of Personality Traits From Body Shapes , 2018, Psychological science.

[136]  Diego Gutierrez,et al.  Perception in graphics, visualization, virtual environments and animation , 2011, SA '11.

[137]  Christian Wallraven,et al.  Understanding and Designing Perceptual Experiments , 2013, Eurographics.

[138]  J. Cutting,et al.  Recognizing friends by their walk: Gait perception without familiarity cues , 1977 .

[139]  Rachel McDonnell,et al.  Facial Features of Non-player Creatures Can Influence Moral Decisions in Video Games , 2017, ACM Trans. Appl. Percept..

[140]  René F. Kizilcec,et al.  The Relationship between Virtual Self Similarity and Social Anxiety , 2014, Front. Hum. Neurosci..

[141]  Ludovic Hoyet,et al.  Evaluating the distinctiveness and attractiveness of human motions on realistic virtual bodies , 2013, ACM Trans. Graph..

[142]  Anthony Steed,et al.  A Comparison of Avatar-, Video-, and Robot-Mediated Interaction on Users’ Trust in Expertise , 2016, Front. Robot. AI.

[143]  Rachel McDonnell,et al.  A perceptual study on the manipulation of facial features for trait portrayal in virtual agents , 2018, IVA.

[144]  J. Kätsyri,et al.  A review of empirical evidence on different uncanny valley hypotheses: support for perceptual mismatch as one road to the valley of eeriness , 2015, Front. Psychol..

[145]  Dana Kulic,et al.  Measurement Instruments for the Anthropomorphism, Animacy, Likeability, Perceived Intelligence, and Perceived Safety of Robots , 2009, Int. J. Soc. Robotics.

[146]  L. Jäncke,et al.  Human Neuroscience , 2022 .

[147]  K. Grammer,et al.  The Reconstruction of Facial Expressions in Embodied Systems New Approaches to an Old Problem , 2006 .

[148]  Angela Tinwell,et al.  Perception of psychopathy and the Uncanny Valley in virtual characters , 2013, Comput. Hum. Behav..

[149]  A. Gilchrist,et al.  An anchoring theory of lightness perception. , 1999 .

[150]  O. John,et al.  Paradigm shift to the integrative Big Five trait taxonomy: History, measurement, and conceptual issues. , 2008 .