VRate: A Unity3D Asset for integrating Subjective Assessment Questionnaires in Virtual Environments

Experience assessment is a key activity for understanding and developing interactive systems and technologies. Virtual reality (VR) is no exception, considering its rising popularity and the increasing availability of affordable VR devices. However, in experience studies where participants typical have to rate multiple test conditions presented to them, having to repeatedly take off the head mounted display (HMD) just for the purpose of issuing ratings is cumbersome and breaks the immersion. As a solution we present VRate, a Unity3D asset to integrate subjective experience questionnaires directly in virtual reality environments. The suitability of VRate has been proven by successfully using the asset in a QoE and a UX study. We describe the technical implementation and suggest how to integrate the VRate in a study setup. VRate will be provided to the scientific community for research purposes, educational and noncommercial usage under a Creative Commons License.

[1]  Nick Bryan-Kinns,et al.  The impact of transitions on user experience in virtual reality , 2017, 2017 IEEE Virtual Reality (VR).

[2]  Xiao Ma,et al.  Web-Based VR Experiments Powered by the Crowd , 2018, WWW.

[3]  Christian Timmerer,et al.  Towards subjective quality of experience assessment for omnidirectional video streaming , 2017, 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX).

[4]  Dieter Kranzlmuller,et al.  State of the art of virtual reality technology , 2016, 2016 IEEE Aerospace Conference.

[5]  Salvatore Livatino,et al.  Handbook for Evaluation Studies in Virtual Reality , 2007, 2007 IEEE Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems.

[6]  Raimund Schatz,et al.  Assessing the QoE Impact of 3D Rendering Style in the Context of VR-based Training , 2018, 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX).

[7]  Helmut Hlavacs,et al.  Evaluating feedback devices for time-continuous mobile multimedia quality assessment , 2014, Signal Process. Image Commun..

[8]  Jonathan Freeman,et al.  A Cross-Media Presence Questionnaire: The ITC-Sense of Presence Inventory , 2001, Presence: Teleoperators & Virtual Environments.

[9]  Ingrid Isenhardt,et al.  Towards Measuring User Experience, Activation and Task Performance in Immersive Virtual Learning Environments for Students , 2016, iLRN.

[10]  Simon Richir,et al.  A questionnaire to measure the user experience in immersive virtual environments , 2016, VRIC.

[11]  Touradj Ebrahimi,et al.  On the performance of objective metrics for omnidirectional visual content , 2017, 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX).

[12]  Manuel Contero,et al.  A Proposal for the Selection of Eye-Tracking Metrics for the Implementation of Adaptive Gameplay in Virtual Reality Based Games , 2017, HCI.

[13]  Deborah Hix,et al.  Usability Evaluation in Virtual Environments: Classification and Comparison of Methods , 2001 .

[14]  Ken Hinckley,et al.  A survey of design issues in spatial input , 1994, UIST '94.