Is Photorealism Important for Perception of Expressive Virtual Humans in Virtual Reality?

In recent years, the quality of real-time rendering has reached new heights—realistic reflections, physically based materials, and photometric lighting are all becoming commonplace in modern game engines and even interactive virtual environments, such as virtual reality (VR). As the strive for realism continues, there is a need to investigate the effect of photorealism on users’ perception, particularly for interactive, emotional scenarios in VR. In this article, we explored three main topics, where we predicted photorealism will make a difference: the illusion of being present with the virtual person and in an environment, altered emotional response toward the character, and a subtler response—comfort of being in close proximity to the character. We present a perceptual experiment, with an interactive expressive virtual character in VR, which was designed to induce particular social responses in people. Our participant pool was large (N = 797) and diverse in terms of demographics. We designed a between-group experiment, where each group saw either the realistic rendering or one of our stylized conditions (simple and sketch style), expressing one of three attitudes: Friendly, Unfriendly, or Sad. While the render style did not particularly effect the level of comfort with the character or increase the illusion of presence with it, our main finding shows that the photorealistic character changed the emotional responses of participants, compared to the stylized versions. We also found a preference for realism in VR, reflected in the affinity and higher place illusion in the scenario, rendered in the realistic render style.

[1]  S. Baron-Cohen,et al.  The Cambridge Mindreading (CAM) Face-Voice Battery: Testing Complex Emotion Recognition in Adults with and without Asperger Syndrome , 2006, Journal of autism and developmental disorders.

[2]  Sara D. Hodges,et al.  Gender Differences, Motivation, and Empathic Accuracy: When it Pays to Understand , 2001 .

[3]  Mel Slater,et al.  The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment , 2003, CHI '03.

[4]  Heloir,et al.  The Uncanny Valley , 2019, The Animation Studies Reader.

[5]  Diego Gutierrez,et al.  To stylize or not to stylize? , 2015, ACM Trans. Graph..

[6]  J. Bailenson,et al.  The Proteus Effect: The Effect of Transformed Self-Representation on Behavior , 2007 .

[7]  Gerd Bruder,et al.  Effects of Unaugmented Periphery and Vibrotactile Feedback on Proxemics with Virtual Humans in AR , 2018, IEEE Transactions on Visualization and Computer Graphics.

[8]  Jeremy N. Bailenson,et al.  Equilibrium Theory Revisited: Mutual Gaze and Personal Space in Virtual Environments , 2001, Presence: Teleoperators & Virtual Environments.

[9]  Mark H. Davis Measuring individual differences in empathy: Evidence for a multidimensional approach. , 1983 .

[10]  James A. Ferwerda,et al.  Three varieties of realism in computer graphics , 2003, IS&T/SPIE Electronic Imaging.

[11]  Heinrich H. Bülthoff,et al.  Render me real? , 2012, ACM Trans. Graph..

[12]  Rachel McDonnell,et al.  The Effect of Realistic Appearance of Virtual Characters in Immersive Environments - Does the Character's Personality Play a Role? , 2018, IEEE Transactions on Visualization and Computer Graphics.

[13]  Karl F. MacDorman,et al.  Too real for comfort? Uncanny responses to computer generated faces , 2009, Comput. Hum. Behav..

[14]  Mel Slater,et al.  The impact of eye gaze on communication using humanoid avatars , 2001, CHI.

[15]  Timothy W. Bickmore,et al.  The Right Agent for the Job? - The Effects of Agent Visual Appearance on Task Domain , 2014, IVA.

[16]  J. Loomis,et al.  Interpersonal Distance in Immersive Virtual Environments , 2003, Personality & social psychology bulletin.

[17]  Mel Slater,et al.  Meeting people vitually: experiments in shared virtual environments , 2002 .

[18]  Andrea Bönsch,et al.  Social VR: How Personal Space is Affected by Virtual Agents' Emotions , 2018, 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[19]  Yuki Yamada,et al.  Categorization difficulty is associated with negative evaluation in the “uncanny valley” phenomenon , 2013 .

[20]  M. Garau,et al.  The Impact of Avatar Fidelity on Social Interaction in Virtual Environments , 2003 .

[21]  R. Michael Latta,et al.  Relation of Status Incongruence to Personal Space , 1978 .

[22]  J. Hodgins,et al.  Anthropomorphism influences perception of computer-animated characters' actions. , 2007, Social cognitive and affective neuroscience.

[23]  Marc Erich Latoschik,et al.  The effect of avatar realism in immersive social virtual realities , 2017, VRST.

[24]  Jeremy N. Bailenson,et al.  The Effect of Behavioral Realism and Form Realism of Real-Time Avatar Faces on Verbal Disclosure, Nonverbal Disclosure, Emotion Recognition, and Copresence in Dyadic Interaction , 2006, PRESENCE: Teleoperators and Virtual Environments.

[25]  Pamela J. Wisniewski,et al.  RealME: the influence of body and hand representations on body ownership and presence , 2017, SUI.

[26]  Susan Persky,et al.  The Independent and Interactive Effects of Embodied-Agent Appearance and Behavior on Self-Report, Cognitive, and Behavioral Markers of Copresence in Immersive Virtual Environments , 2005, Presence: Teleoperators & Virtual Environments.

[27]  Mel Slater,et al.  Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[28]  Rachel McDonnell,et al.  Don't stand so close to me: investigating the effect of control on the appeal of virtual humans using immersion and a proximity-based behavioral task , 2017, SAP.

[29]  Sabarish V. Babu,et al.  Effects of Virtual Human Appearance Fidelity on Emotion Contagion in Affective Inter-Personal Simulations , 2016, IEEE Transactions on Visualization and Computer Graphics.

[30]  D. Watson,et al.  Development and validation of brief measures of positive and negative affect: the PANAS scales. , 1988, Journal of personality and social psychology.

[31]  Mary C. Whitton,et al.  A Psychophysical Experiment Regarding Components of the Plausibility Illusion , 2017, IEEE Transactions on Visualization and Computer Graphics.

[32]  Jessica K. Hodgins,et al.  Unpleasantness of animated characters corresponds to increased viewer attention to faces , 2013, SAP.

[33]  Philip W. Fink,et al.  Obstacle avoidance during walking in real and virtual environments , 2007, TAP.

[34]  Ulrike Schultze,et al.  Embodiment and presence in virtual worlds: a review , 2010, J. Inf. Technol..

[35]  Sophie Jörg,et al.  Need a hand?: how appearance affects the virtual hand illusion , 2016, SAP.

[36]  J. Blascovich Social influence within immersive virtual environments , 2002 .

[37]  Jessica K. Hodgins,et al.  Perceptual effects of damped and exaggerated facial motion in animated characters , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[38]  Matthias Scheutz,et al.  A mismatch in the human realism of face and voice produces an uncanny valley , 2011, i-Perception.