The effects of 3D sound in a 360-degree live concert video on social presence, parasocial interaction, enjoyment, and intent of financial supportive action

Abstract What would be the social consequences of rendering three-dimensional (3D) sound in a pre-recorded live concert video? Although previous studies demonstrate that the rendering of 3D sound in a pre-recorded live concert may foster a sense of presence, whether and how 3D sound will affect audience's experiences beyond the reporting of a sense of presence has yet to be elucidated. To further investigate the socio-psychological impact of 3D sound in a pre-recorded live concert content, a 2 × 2 mixed design experiment (N = 44), having sound as a within factor (2D vs. 3D) and visualization display mode as a between factor (monoscopic TV vs. stereoscopic VR), was conducted. Drawing from the social presence theory and related work, we specifically investigated whether 3D sound in a pre-recorded live concert content, rendered either through a monoscopic TV screen or a stereoscopic VR device, will contribute to enhancing audience's sense of being with a performer (i.e., social presence), and whether this will subsequently influence audience's psychological engagement with the performer (i.e., parasocial interaction), enjoyment as need satisfaction, and intent of financial supportive action towards the performer. Results showed that 3D sound enhanced a sense of social presence, and this, in turn, positively influenced parasocial interaction and enjoyment. In addition, parasocial interaction, fostered by social presence, positively affected audience's intent of financial supportive action. Further implications of the findings with respect to the role of 3D sound in a pre-recorded live concert content, rendered through a VR technology, are discussed.

[1]  M. Rosa,et al.  Presence and emotions in virtual environments: the influence of stereoscopy. , 2008 .

[2]  Mark Shevy,et al.  Music genre as cognitive schema: extramusical associations with country and hip-hop music , 2008 .

[3]  Izak Benbasat,et al.  Addressing the What and How of Online Services: Positioning Supporting-Services Functionality and Service Quality for Business-to-Consumer Success , 2008, Inf. Syst. Res..

[4]  D. Giles.  Parasocial Interaction: A Review of the Literature and a Model for Future Research , 2002 .

[5]  Timo Peltonen,et al.  A Rehearsal Hall with Virtual Acoustics for Symphony Orchestras , 2009 .

[6]  Y. Ikei,et al.  Texture presentation by vibratory tactile display-image based presentation of a tactile texture , 1997, Proceedings of IEEE 1997 Annual International Symposium on Virtual Reality.

[7]  Eun-Ju Lee,et al.  Not So Imaginary Interpersonal Contact With Public Figures on Social Network Sites , 2013, Commun. Res..

[8]  Seung-A. Annie Jin,et al.  Parasocial Interaction with an Avatar in Second Life: A Typology of the Self and an Empirical Test of the Mediating Role of Social Presence , 2010, PRESENCE: Teleoperators and Virtual Environments.

[9]  S. Shyam Sundar,et al.  Mobile Persuasion: Can Screen Size and Presentation Mode Make a Difference to Trust? , 2016 .

[10]  Jack M. Loomis,et al.  Visual perception of egocentric distance in real and virtual environments. , 2003 .

[11]  Daniela Massiceti,et al.  Stereosonic vision: Exploring visual-to-auditory sensory substitution mappings in an immersive virtual reality navigation paradigm , 2018, PloS one.

[12]  Dianne Cyr,et al.  The role of social presence in establishing loyalty in e-Service environments , 2007, Interact. Comput..

[13]  M. Lombard,et al.  Presence and Television: The Role of Screen Size. , 2000 .

[14]  P. M. Podsakoff,et al.  Self-Reports in Organizational Research: Problems and Prospects , 1986 .

[15]  Karen Collins,et al.  Making gamers cry: mirror neurons and embodied interaction with game sound , 2011, AM '11.

[16]  Matthew Lombard,et al.  At the Heart of It All: The Concept of Presence , 2006 .

[17]  A. Rubin,et al.  Audience Activity and Soap Opera Involvement: A Uses and Effects Investigation. , 1987 .

[18]  Lawrence W. Stark,et al.  The Effects of Pictorial Realism, Delay of Visual Feedback, and Observer Interactivity on the Subjective Sense of Presence , 1996, Presence: Teleoperators & Virtual Environments.

[19]  Gang Ren,et al.  Towards the Design of Effective Haptic and Audio Displays for Augmented Reality and Mixed Reality Applications , 2018, Adv. Multim..

[20]  R. Donovan,et al.  Self-determined motivation in sport predicts anti-doping motivation and intention: a perspective from the trans-contextual model. , 2014, Journal of science and medicine in sport.

[21]  Hans van Buuren,et al.  Determining Sociability, Social Space, and Social Presence in (A)synchronous Collaborative Groups , 2004, Cyberpsychology Behav. Soc. Netw..

[22]  Michael J. Singer,et al.  Measuring Presence in Virtual Environments: A Presence Questionnaire , 1998, Presence.

[23]  Albert A. Rizzo,et al.  Immersiveness and Physiological Arousal within Panoramic Video-Based Virtual Reality , 2007, Cyberpsychology Behav. Soc. Netw..

[24]  Amanda K Montoya,et al.  Two-condition within-participant statistical mediation analysis: A path-analytic framework. , 2017, Psychological methods.

[25]  R H Y So,et al.  Toward orthogonal non-individualised head-related transfer functions for forward and backward directional sound: cluster analysis and an experimental study , 2010, Ergonomics.

[26]  T. Eerola.  Are the Emotions Expressed in Music Genre-specific? An Audio-based Evaluation of Datasets Spanning Classical, Film, Pop and Mixed Genres , 2011 .

[27]  Maria V. Sanchez-Vives,et al.  From presence to consciousness through virtual reality , 2005, Nature Reviews Neuroscience.

[28]  Juho Hamari,et al.  hy do people buy virtual goods ? Attitude toward virtual good urchases versus game enjoyment , 2015 .

[29]  Christopher R Gibson,et al.  Sound Tracks: Popular Music Identity and Place , 2002 .

[30]  Daniel Västfjäll,et al.  Better Presence and Performance in Virtual Environments by Improved Binaural Sound Rendering , 2002 .

[31]  Mendel Kleiner,et al.  Auditory Presence, Individualized Head-Related Transfer Functions, and Illusory Ego-Motion in Virtual Environments , 2004 .

[32]  Richard L. Daft,et al.  Message Equivocality, Media Selection, and Manager Performance: Implications for Information Systems , 1987, MIS Q..

[33]  William L. Martens,et al.  Perceptual evaluation of filters controlling source direction : Customized and generalized HRTFs for binaural synthesis , 2003 .

[34]  Dominique Bechmann,et al.  Effects of stereo and head tracking on distance estimation, presence, and simulator sickness using wall screen in architectural project review , 2017, 2017 IEEE Symposium on 3D User Interfaces (3DUI).

[35]  Jihye Park,et al.  Television Apparel Shopping: Impulse Buying and Parasocial Interaction , 2004 .

[36]  Larry F. Hodges,et al.  Can a virtual cat persuade you?: the role of gender and realism in speaker persuasiveness , 2006, CHI.

[37]  A. Rubin,et al.  LONELINESS, PARASOCIAL INTERACTION, AND LOCAL TELEVISION NEWS VIEWING , 1985 .

[38]  Jan Noyes,et al.  An assessment of the influence of perceived enjoyment and attitude on the intention to use technology among pre-service teachers: A structural equation modeling approach , 2011, Comput. Educ..

[39]  Donghee Yvette Wohn,et al.  Explaining Viewers' Emotional, Instrumental, and Financial Support Provision for Live Streamers , 2018, CHI.

[40]  Ron Tamborini,et al.  Defining Media Enjoyment as the Satisfaction of Intrinsic Needs , 2010 .

[41]  Sungyoung Kim,et al.  Listeners' Response to String Quartet Performances Recorded in Virtual Acoustics , 2015 .

[42]  Mendel Kleiner,et al.  Auralization-An Overview , 1993 .

[43]  Frank Biocca,et al.  Toward a More Robust Theory and Measure of Social Presence: Review and Suggested Criteria , 2003, Presence: Teleoperators & Virtual Environments.

[44]  Peter Shirley,et al.  Visual cues for imminent object contact in realistic virtual environments , 2000 .

[45]  Kenji Ozawa,et al.  Psychological factors involved in auditory presence , 2003 .

[46]  N. Alperstein.  Imaginary social relationships with celebrities appearing in television commercials , 1991 .

[47]  Norazah Mohd Suki,et al.  Exploring the relationship between perceived usefulness, perceived ease of use, perceived enjoyment, attitude and subscribers' intention towards using 3G mobile services , 2011 .

[48]  Woodrow Barfield,et al.  The Sense of Presence within Auditory Virtual Environments , 1996, Presence: Teleoperators & Virtual Environments.

[49]  Paul Benjamin Lowry,et al.  The CMC Interactivity Model: How Interactivity Enhances Communication Quality and Process Satisfaction in Lean-Media Groups , 2009, J. Manag. Inf. Syst..

[50]  D. A. Kenny,et al.  Estimating and testing mediation and moderation in within-subject designs. , 2001, Psychological methods.

[51]  T. Kuhlen,et al.  Does the Directivity of a Virtual Agent's Speech Influence the Perceived Social Presence? , 2018 .

[52]  Durand R. Begault,et al.  3-D Sound for Virtual Reality and Multimedia Cambridge , 1994 .

[53]  Matthew K. O. Lee,et al.  Exploring consumers' impulse buying behavior on social commerce platform: The role of parasocial interaction , 2016, Int. J. Inf. Manag..

[54]  D. Horton,et al.  Mass communication and para-social interaction; observations on intimacy at a distance. , 1956, Psychiatry.

[55]  Jonathan Steuer,et al.  Defining virtual reality: dimensions determining telepresence , 1992 .

[56]  F. J. Brewerton,et al.  Rock on! An elementary model of customer satisfaction with musical performances , 2004 .

[57]  Hayeon Song,et al.  Celebrity’s Self-Disclosure on Twitter and Parasocial Relationships: A Mediating Role of Social Presence , 2016 .

[58]  Richard L. Daft,et al.  Organizational information requirements, media richness and structural design , 1986 .

[59]  John Short,et al.  The social psychology of telecommunications , 1976 .

[60]  Michael Vorlaender,et al.  Virtual acoustics: opportunities and limits of spatial soundreproduction , 2008 .

[61]  Eun Ju Lee.  Effectiveness of Politicians' Soft Campaign on Twitter Versus TV: Cognitive and Experiential Routes , 2013 .

[62]  Björn Niehaves,et al.  Designing electronic feedback - Analyzing the effects of social presence on perceived feedback usefulness , 2015, Int. J. Hum. Comput. Stud..

[63]  Vanessa Evers,et al.  The influence of social presence on acceptance of a companion robot by older people , 2008 .

[64]  Michael Vorlnder,et al.  Auralization: Fundamentals of Acoustics, Modelling, Simulation, Algorithms and Acoustic Virtual Reality , 2020 .

[65]  Clifford Nass,et al.  Designing social presence of social actors in human computer interaction , 2003, CHI '03.

[66]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[67]  Mark A. Fox,et al.  Concert Tour Success in North America: An Examination of the Top 100 Tours from 1997 to 2005 , 2007 .

[68]  Frank Biocca,et al.  Immersive virtual reality technology , 1995 .

[69]  Maria V. Sanchez-Vives,et al.  First Person Experience of Body Transfer in Virtual Reality , 2010, PloS one.

[70]  Gerard Jounghyun Kim,et al.  A SWOT Analysis of the Field of Virtual Reality Rehabilitation and Therapy , 2005, Presence: Teleoperators & Virtual Environments.

[71]  Namkee Park,et al.  Effects of pre-game stories on feelings of presence and evaluation of computer games , 2010, Int. J. Hum. Comput. Stud..

[72]  Paul M. Hofman,et al.  Relearning sound localization with new ears , 1998, Nature Neuroscience.

[73]  Daniel Västfjäll,et al.  ECOLOGICAL ACOUSTICS AND THE MULTI-MODAL PERCEPTION OF ROOMS: REAL AND UNREAL EXPERIENCES OF AUDITORY-VISUAL VIRTUAL ENVIRONMENTS , 2001 .

[74]  C. Heeter.  Communication research on consumer VR , 1995 .

[75]  Mel Slater,et al.  The Plausibility of a String Quartet Performance in Virtual Reality , 2017, IEEE Transactions on Visualization and Computer Graphics.