Perceiving attitudes expressed through nonverbal behaviors in immersive virtual environments

Virtual Reality and immersive experiences, which allow players to share the same virtual environment as the characters of a virtual world, have gained more and more interest recently. In order to conceive these immersive virtual worlds, one of the challenges is to give to the characters that populate them the ability to express behaviors that can support the immersion. In this work, we propose a model capable of controlling and simulating a conversational group of social agents in an immersive environment. We describe this model which has been previously validated using a regular screen setting and we present a study for measuring whether users recognized the attitudes expressed by virtual agents through the realtime generated animations of nonverbal behavior in an immersive setting. Results mirrored those of the regular screen setting thus providing further insights for improving players experiences by integrating them into immersive simulated group conversations with characters that express different interpersonal attitudes.

[1]  Marc Cavazza,et al.  Immersive FPS games: user experience and performance , 2013, ImmersiveMe '13.

[2]  K. Scherer What are emotions? And how can they be measured? , 2005 .

[3]  E. Goffman,et al.  Forms of talk , 1982 .

[4]  Catherine Pelachaud,et al.  From a User-created Corpus of Virtual Agent's Non-verbal Behavior to a Computational Model of Interpersonal Attitudes , 2013, IVA.

[5]  Katherine R. Storrs,et al.  Sociality of facial expressions in immersive virtual environments: A facial EMG study , 2012, Biological Psychology.

[6]  Doug A. Bowman,et al.  Evaluating Display Fidelity and Interaction Fidelity in a Virtual Reality Game , 2012, IEEE Transactions on Visualization and Computer Graphics.

[7]  Henrik M. Peperkorn,et al.  The impact of perception and presence on emotional reactions: a review of research in virtual reality , 2015, Front. Psychol..

[8]  Kristinn R. Thórisson,et al.  A Multiparty Multimodal Architecture for Realtime Turntaking , 2010, IVA.

[9]  Catherine Pelachaud,et al.  Suggestions for Extending SAIBA with the VIB Platform , 2014 .

[10]  Nikos Papadopoulos,et al.  Full immersive virtual environment CAVETM in chemistry education , 2008, Comput. Educ..

[11]  Nadia Magnenat-Thalmann,et al.  Modelling Multi-Party Interactions among Virtual Characters, Robots, and Humans , 2014, PRESENCE: Teleoperators and Virtual Environments.

[12]  E. Hall,et al.  The Hidden Dimension , 1970 .

[13]  Stephen Travis Pope,et al.  The AlloSphere: Immersive Multimedia for Scientific Discovery and Artistic Exploration , 2009, IEEE MultiMedia.

[14]  Marta Kristín Lárusdóttir,et al.  Avatars in Conversation: The Importance of Simulating Territorial Behavior , 2010, IVA.

[15]  Jeremy N. Bailenson,et al.  How Immersive Is Enough? A Meta-Analysis of the Effect of Immersive Technology on User Presence , 2016 .

[16]  Ruzena Bajcsy,et al.  The Effects of Fully Immersive Virtual Reality on the Learning of Physical Tasks , 2006 .

[17]  Catherine Pelachaud,et al.  Conversational Behavior Reflecting Interpersonal Attitudes in Small Group Interactions , 2015, IVA.

[18]  J. Bailenson,et al.  Virtual Reality A Survival Guide for the Social Scientist , 1998 .

[19]  Dwi Hartanto,et al.  Design and Implementation of Home-Based Virtual Reality Exposure Therapy System with a Virtual eCoach , 2015, IVA.

[20]  Carolina Cruz-Neira,et al.  Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE , 2023 .

[21]  J. V. Bradley Complete Counterbalancing of Immediate Sequential Effects in a Latin Square Design , 1958 .

[22]  D. Funder,et al.  The Riverside Behavioral Q-sort: a tool for the description of social behavior. , 2000, Journal of Personality.

[23]  A. Kendon Conducting Interaction: Patterns of Behavior in Focused Encounters , 1990 .

[24]  Jeremy N. Bailenson,et al.  Equilibrium Theory Revisited: Mutual Gaze and Personal Space in Virtual Environments , 2001, Presence: Teleoperators & Virtual Environments.

[25]  Arsalan Heydarian,et al.  Immersive Virtual Environments: Experiments on Impacting Design and Human Building Interaction , 2014, CAADRIA proceedings.

[26]  Dario Bombari,et al.  Studying social interactions through immersive virtual environment technology: virtues, pitfalls, and future challenges , 2015, Front. Psychol..