Crowdsourcing Virtual Reality Experiments using VRChat

Research involving Virtual Reality (VR) headsets is becoming more and more popular. However, scaling VR experiments is challenging as researchers are often limited to using one or a small number of headsets for in-lab studies. One general way to scale experiments is through crowdsourcing so as to have access to a large pool of diverse participants with relatively little expense of time and money. Unfortunately, there is no easy way to crowdsource VR experiments. We demonstrate that it is possible to implement and run crowdsourced VR experiments using a pre-existing massively multiplayer online VR social platform - VRChat. Our small (n=10) demonstration experiment required participants to navigate a maze in VR. Participants searched for two targets then returned to the exit while we captured completion time and position over time. While there are some limitations with using VRChat, overall we have demonstrated a promising approach for running crowdsourced VR experiments.

[1]  Xiao Ma,et al.  Web-Based VR Experiments Powered by the Crowd , 2018, WWW.

[2]  Yalong Yang,et al.  Maps and Globes in Virtual Reality , 2018, Comput. Graph. Forum.

[3]  Panagiotis G. Ipeirotis,et al.  Running Experiments on Amazon Mechanical Turk , 2010, Judgment and Decision Making.

[4]  A. Acquisti,et al.  Beyond the Turk: Alternative Platforms for Crowdsourcing Behavioral Research , 2016 .

[5]  Kasper Hornbæk,et al.  Virtual reality studies outside the laboratory , 2017, VRST.

[6]  Aniket Kittur,et al.  Crowdsourcing user studies with Mechanical Turk , 2008, CHI.

[7]  Anthony Steed,et al.  A longitudinal study of small group interaction in social virtual reality , 2018, VRST.

[8]  Emanuele Palazzetti Getting Started with UDOO , 2015 .

[9]  David G. Rand,et al.  The promise of Mechanical Turk: how online labor markets can help theorists run behavioral experiments. , 2012, Journal of theoretical biology.

[10]  Jeffrey Heer,et al.  Crowdsourcing graphical perception: using mechanical turk to assess visualization design , 2010, CHI.

[11]  Bill Tomlinson,et al.  Who are the crowdworkers?: shifting demographics in mechanical turk , 2010, CHI Extended Abstracts.

[12]  Bernd Huber,et al.  Conducting online virtual environment experiments with uncompensated, unsupervised samples , 2020, PloS one.

[13]  Michael D. Buhrmester,et al.  Amazon's Mechanical Turk , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[14]  A. James 2010 , 2011, Philo of Alexandria: an Annotated Bibliography 2007-2016.

[15]  Jesse Chandler,et al.  Using Mechanical Turk to Study Clinical Populations , 2013 .

[16]  Kyle Johnsen,et al.  Navigating a maze differently - a user study , 2018, ArXiv.