Users’ psychophysiological, vocal, and self-reported responses to the apparent attitude of a virtual audience in stereoscopic 360°-video

This research analyzes the psychological reactions of participants to the neutral, positive, or negative attitudes of a virtual audience recorded in 360°-video. Participants were asked to deliver three speeches, each accompanied by a different type of reaction from the virtual audience. Measures of user state included questionnaires, psychophysiological measures, and voice recordings. The results showed that, compared to the neutral audience, the negative audience elicited increases in skin conductance level and heart rate variability, decreases in voice intensity, and a higher ratio of silent parts in the speech, as well as a more negative self-reported valence, higher anxiety, and lower social presence. These findings evidence that, even if some attributes that are considered to be central to immersive experiences are lacking, (i.e., interactivity or avatar representation of the user), 360°-video recreations of virtual environments lead to realistic reactions on users. These results support the effectiveness of 360°-video virtual audiences for public speaking training and social anxiety treatment.

[1]  Kay Connelly,et al.  Toward total quality of experience: A QoE model in a communication ecosystem , 2012, IEEE Communications Magazine.

[2]  Ulrich Frank,et al.  Multilevel Modeling , 2014, Business & Information Systems Engineering.

[3]  Luca Citi,et al.  cvxEDA: A Convex Optimization Approach to Electrodermal Activity Processing , 2016, IEEE Transactions on Biomedical Engineering.

[4]  C. Erik Timmerman,et al.  Effects of Supportive Feedback Messages on Exergame Experiences: A Mediating Role of Social Presence , 2018, J. Media Psychol. Theor. Methods Appl..

[5]  P. Emmelkamp,et al.  Can virtual reality exposure therapy gains be generalized to real-life? A meta-analysis of studies applying behavioral assessments. , 2015, Behaviour research and therapy.

[6]  Peter Robinson,et al.  Cognition and Second Language Instruction: Task complexity, cognitive resources, and syllabus design: a triadic framework for examining task influences on SLA , 2001 .

[7]  Wijnand A. IJsselsteijn,et al.  Using Behavioral Realism to Estimate Presence: A Study of the Utility of Postural Responses to Motion Stimuli , 2000, Presence: Teleoperators & Virtual Environments.

[8]  Mariano Alcañiz Raya,et al.  Affective Interactions Using Virtual Reality: The Link between Presence and Emotions , 2007, Cyberpsychology Behav. Soc. Netw..

[9]  Mtm Marc Lambooij,et al.  Visual Discomfort and Visual Fatigue of Stereoscopic Displays: A Review , 2009 .

[10]  Paul Boersma,et al.  Praat: doing phonetics by computer , 2003 .

[11]  Rosa María Baños,et al.  Immersion and Emotion: Their Impact on the Sense of Presence , 2004, Cyberpsychology Behav. Soc. Netw..

[12]  Eric L Groen,et al.  The effect of internal and external fields of view on visually induced motion sickness. , 2010, Applied ergonomics.

[13]  Kay M. Stanney,et al.  What to Expect from Immersive Virtual Environment Exposure: Influences of Gender, Body Mass Index, and Past Experience , 2003, Hum. Factors.

[14]  K. Vohs,et al.  Case Western Reserve University , 1990 .

[15]  M. Dawson,et al.  The electrodermal system , 2007 .

[16]  Brenda K. Wiederhold,et al.  Virtual reality therapy for anxiety disorders: Advances in evaluation and treatment. , 2005 .

[17]  T. Hartmann The Spatial Presence Experience Scale (spes): a Short Self-report Measure for Diverse Media Settings the Spatial Presence Experience Scale (spes): a Short Self-report Measure for Diverse Media Settings Post-print Version , 2022 .

[18]  Mats Fredrikson,et al.  In a Nervous Voice: Acoustic Analysis and Perception of Anxiety in Social Phobics’ Speech , 2008 .

[19]  G. Bui,et al.  L2 fluency as influenced by content familiarity and planning: Performance, measurement, and pedagogy , 2018 .

[20]  Jim Blascovich,et al.  Social evaluations of embodied agents and avatars , 2011, Comput. Hum. Behav..

[21]  Mel Slater,et al.  Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[22]  Nicola Döring,et al.  Designing Virtual Audiences for Fear of Public Speaking Training - An Observation Study on Realistic Nonverbal Behavior , 2012, Annual Review of Cybertherapy and Telemedicine.

[23]  Mel Slater,et al.  Public Speaking in Virtual Reality: Facing an Audience of Avatars , 1999, IEEE Computer Graphics and Applications.

[24]  Hany Ferdinando,et al.  Emotion Recognition using cvxEDA-Based Features , 2018 .

[25]  Marnix S. van Gisbergen,et al.  Beat the Fear of Public Speaking: Mobile 360° Video Virtual Reality Exposure Training in Home Environment Reduces Public Speaking Anxiety , 2017, Cyberpsychology Behav. Soc. Netw..

[26]  Christoph Klimmt,et al.  A Process Model of the Formation of Spatial Presence Experiences , 2007 .

[27]  M. Birna van Riemsdijk,et al.  The design of virtual audiences: Noticeable and recognizable behavioral styles , 2016, Comput. Hum. Behav..

[28]  M. Craske,et al.  Optimizing inhibitory learning during exposure therapy. , 2008, Behaviour research and therapy.

[29]  Mary C. Whitton,et al.  A Psychophysical Experiment Regarding Components of the Plausibility Illusion , 2017, IEEE Transactions on Visualization and Computer Graphics.

[30]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[31]  N. Ravaja Contributions of Psychophysiology to Media Research: Review and Recommendations , 2004 .

[32]  Maria V. Sanchez-Vives,et al.  A Virtual Reprise of the Stanley Milgram Obedience Experiments , 2006, PloS one.

[33]  Joseph J. LaViola,et al.  A discussion of cybersickness in virtual environments , 2000, SGCH.

[34]  Mel Slater,et al.  Simulating virtual environments within virtual environments as the basis for a psychophysics of presence , 2010, ACM Trans. Graph..

[35]  J. Bailenson,et al.  Virtual Humans and Persuasion: The Effects of Agency and Behavioral Realism , 2007 .

[36]  C. Botella,et al.  Virtual reality exposure therapy in anxiety disorders: a quantitative meta‐analysis , 2012, Depression and anxiety.

[37]  F. Biocca,et al.  Criteria And Scope Conditions For A Theory And Measure Of Social Presence , 2001 .

[38]  Max M. North,et al.  Brief Virtual Reality Therapy for Public Speaking Anxiety , 2002, Cyberpsychology Behav. Soc. Netw..

[39]  Jeremy N. Bailenson,et al.  How Immersive Is Enough? A Meta-Analysis of the Effect of Immersive Technology on User Presence , 2016 .

[40]  F. Bonato,et al.  Combined pitch and roll and cybersickness in a virtual environment. , 2009, Aviation, space, and environmental medicine.

[41]  Daniel Mestre,et al.  Afforded actions as a behavioral assessment of physical presence in virtual environments , 2009, Virtual Reality.

[42]  Marcus Barkowsky,et al.  How visual fatigue and discomfort impact 3D-TV quality of experience: a comprehensive review of technological, psychophysical, and psychological factors , 2013, annals of telecommunications - annales des télécommunications.

[43]  Mel Slater,et al.  The Use of Questionnaire Data in Presence Studies: Do Not Seriously Likert , 2007, PRESENCE: Teleoperators and Virtual Environments.

[44]  Helmut Hlavacs,et al.  A Virtual Training Tool for Giving Talks , 2012, ICEC.

[45]  J. Bailenson,et al.  Comparing Behavioral and Self-Report Measures of Embodied Agents' Social Presence in Immersive Virtual Environments , 2004 .

[46]  Xueni Pan,et al.  Socially Anxious and Confident Men Interact with a Forward Virtual Woman: An Experimental Study , 2012, PloS one.

[47]  David M. Hoffman,et al.  Vergence-accommodation conflicts hinder visual performance and cause visual fatigue. , 2008, Journal of vision.

[48]  Xuyan Qiu Functions of oral monologic tasks: Effects of topic familiarity on L2 speaking performance: , 2019 .

[49]  Timothy D. Wilson,et al.  The halo effect: Evidence for unconscious alteration of judgments. , 1977 .

[50]  W. Roth,et al.  The somatic symptom paradox in DSM-IV anxiety disorders: suggestions for a clinical focus in psychophysiology , 2001, Biological Psychology.

[51]  Mel Slater,et al.  How Colorful Was Your Day? Why Questionnaires Cannot Assess Presence in Virtual Environments , 2004, Presence: Teleoperators & Virtual Environments.

[52]  L. Carretié Exogenous (automatic) attention to emotional stimuli: a review , 2014, Cognitive, Affective, & Behavioral Neuroscience.

[53]  S. Hofmann,et al.  Speech disturbances and gaze behavior during public speaking in subtypes of social phobia. , 1997, Journal of anxiety disorders.

[54]  Danielle Oprean,et al.  Being There in the Midst of the Story: How Immersive Journalism Affects Our Perceptions and Cognitions , 2017, Cyberpsychology Behav. Soc. Netw..

[55]  P. Anderson,et al.  Virtual Reality and Exposure Group Therapy for Social Anxiety Disorder: Results from a 4–6 Year Follow-Up , 2017, Cognitive Therapy and Research.

[56]  Federica Battisti,et al.  Toward the Assessment of Quality of Experience for Asymmetric Encoding in Immersive Media , 2018, IEEE Transactions on Broadcasting.

[57]  P. Lang The emotion probe. Studies of motivation and attention. , 1995, The American psychologist.

[58]  Mel Slater,et al.  Enhancing Our Lives with Immersive Virtual Reality , 2016, Front. Robot. AI.

[59]  Sandra Poeschl,et al.  Virtual Reality Training for Public Speaking—A QUEST-VR Framework Validation , 2017, Front. ICT.

[60]  Carey K. Morewedge,et al.  Associative processes in intuitive judgment , 2010, Trends in Cognitive Sciences.

[61]  W. Revelle,et al.  Coefficients Alpha, Beta, Omega, and the glb: Comments on Sijtsma , 2009 .

[62]  Annie Lang,et al.  WHERE PSYCHOPHYSIOLOGY MEETS THE MEDIA: Taking the Effects Out of Mass Media Research , 2009 .

[63]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[64]  Isabel L. Kampmann,et al.  Controlling Social Stress in Virtual Reality Environments , 2014, PloS one.

[65]  René F. Kizilcec,et al.  The Relationship between Virtual Self Similarity and Social Anxiety , 2014, Front. Hum. Neurosci..

[66]  Chris Barker,et al.  An Experimental Study on Fear of Public Speaking Using a Virtual Environment , 2006, Cyberpsychology Behav. Soc. Netw..

[67]  Eugene Nalivaiko,et al.  A comparative study of cybersickness during exposure to virtual reality and "classic" motion sickness: are they different? , 2018, Journal of applied physiology.

[68]  Peter Schelkens,et al.  Qualinet White Paper on Definitions of Quality of Experience , 2013 .

[69]  Chris Barker,et al.  An Experiment on Public Speaking Anxiety in Response to Three Different Types of Virtual Audience , 2002, Presence: Teleoperators & Virtual Environments.

[70]  S. Delacroix,et al.  The Responses of Medical General Practitioners to Unreasonable Patient Demand for Antibiotics - A Study of Medical Ethics Using Immersive Virtual Reality , 2016, PloS one.

[71]  K. Scherer Psychological models of emotion. , 2000 .

[72]  S. Dickerson,et al.  Acute stressors and cortisol responses: a theoretical integration and synthesis of laboratory research. , 2004, Psychological bulletin.

[73]  J. Thayer,et al.  Heart Rate Variability and Cardiac Vagal Tone in Psychophysiological Research – Recommendations for Experiment Planning, Data Analysis, and Data Reporting , 2017, Front. Psychol..

[74]  Peter J. Lang,et al.  Fear reduction and fear behavior: Problems in treating a construct. , 1968 .