Conducting online virtual environment experiments with uncompensated, unsupervised samples

Web-based experimentation with uncompensated and unsupervised samples allows for a larger and more diverse sample population, more generalizable results, and faster theory to experiment cycle. Given that participants are unsupervised, it is still unknown whether the data collected in such settings would be of sufficiently high quality to support robust conclusions. Therefore, we investigated the feasibility of conducting such experiments online using virtual environment technologies. We conducted a conceptual replication of two prior experiments that have been conducted in virtual environments. Our results replicate findings previously obtained in conventional laboratory settings. These results hold across different device types of participants (ranging from desktop, through mobile devices to immersive virtual reality headsets), suggesting that experiments can be conducted online with uncompensated samples in virtual environments.

[1]  M. Nowak,et al.  Fairness versus reason in the ultimatum game. , 2000, Science.

[2]  Crystal L. Hoyt,et al.  Immersive Virtual Environment Technology as a Methodological Tool for Social Psychology , 2002 .

[3]  Jeremy N. Bailenson,et al.  Immersion at scale: Researcher's guide to ecologically valid mobile experiments , 2016, 2016 IEEE Virtual Reality (VR).

[4]  Siddharth Suri,et al.  Conducting behavioral research on Amazon’s Mechanical Turk , 2010, Behavior research methods.

[5]  Mary Czerwinski,et al.  Immersion in desktop virtual reality , 1997, UIST '97.

[6]  Jeremy N. Bailenson,et al.  The embodiment of sexualized virtual selves: The Proteus effect and experiences of self-objectification via avatars , 2013, Comput. Hum. Behav..

[7]  Aniket Kittur,et al.  An Assessment of Intrinsic and Extrinsic Motivation on Task Performance in Crowdsourcing Markets , 2011, ICWSM.

[8]  Chris Dede,et al.  Many ways to walk a mile in another's moccasins: Type of social perspective taking and its effect on negotiation outcomes , 2015, Comput. Hum. Behav..

[9]  Xiao Ma,et al.  Web-Based VR Experiments Powered by the Crowd , 2018, WWW.

[10]  Katharina Reinecke,et al.  Crowdsourcing performance evaluations of user interfaces , 2013, CHI.

[11]  G. Evans,et al.  Restorative Effects of Natural Environment Experiences , 1991 .

[12]  J. Bailenson,et al.  Transformed Representation 1 RUNNING HEAD: TRANSFORMED REPRESENTATION The Proteus Effect: Implications of Transformed Digital Self-Representation on Online and Offline Behavior , 2008 .

[13]  S Nichols,et al.  Physical ergonomics of virtual environment use. , 1999, Applied ergonomics.

[14]  Katharina Reinecke,et al.  LabintheWild: Conducting Large-Scale Online Experiments With Uncompensated Samples , 2015, CSCW.

[15]  Kasper Hornbæk,et al.  Virtual reality studies outside the laboratory , 2017, VRST.

[16]  M. Banaji,et al.  Psychological. , 2015, The journals of gerontology. Series B, Psychological sciences and social sciences.

[17]  J. Bailenson,et al.  The Proteus Effect: The Effect of Transformed Self-Representation on Behavior , 2007 .

[18]  Hunter G Hoffman,et al.  Virtual reality pain control during burn wound debridement of combat-related burn injuries using robot-like arm mounted VR goggles. , 2011, The Journal of trauma.

[19]  David Swapp,et al.  An ‘In the Wild’ Experiment on Presence and Embodiment using Consumer Virtual Reality Equipment , 2016, IEEE Transactions on Visualization and Computer Graphics.

[20]  Raz Schwartz,et al.  All are welcome: Using VR ethnography to explore harassment behavior in immersive social virtual reality , 2017, 2017 IEEE Virtual Reality (VR).

[21]  Brandon Van Der Heide,et al.  The Proteus Effect in Dyadic Communication , 2013, Commun. Res..

[22]  K. Nakayama,et al.  Is the Web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments , 2012, Psychonomic Bulletin & Review.

[23]  Katharina Reinecke,et al.  The Effect of Performance Feedback on Social Media Sharing at Volunteer-Based Online Experiment Platforms , 2017, CHI.

[24]  Jeremy N. Bailenson,et al.  How Immersive Is Enough? A Meta-Analysis of the Effect of Immersive Technology on User Presence , 2016 .

[25]  J. Lanier,et al.  Model of Illusions and Virtual Reality , 2017, Front. Psychol..

[26]  S. Moffat,et al.  Navigation in a “Virtual” Maze: Sex Differences and Correlation With Psychometric Measures of Spatial Ability in Humans , 1998 .

[27]  N BailensonJeremy,et al.  Many ways to walk a mile in another's moccasins , 2015 .