Understanding VR Software Testing Needs from Stakeholders’ Points of View

Software testing is a critical activity to ensure that software complies with its specification and guarantee for delivering high quality products. However, there are a number of challenges to be faced as new software domains are created. Although there are some works in the literature that explore the application of software testing practices specific to the context of Virtual Reality (VR), these works tend to reflect only the researcher’s point of view, leaving aside the players involved in the VR development process, contributing to a gap that makes it difficult to popularize such practices. Aiming to reduce this gap, this paper presents an analysis under the perception of stakeholders involved in the process of developing and using VR applications. We conducted a survey with 88 respondents, comprising different roles, interviewed during a virtual reality scientific conference and later complemented by electronic dissemination, reaching five different profiles. The results showed that there is a major concern among all the interviewed profiles regarding the impact of faults in VR applications. In addition, we investigated these results in both software and hardware aspects. The results related to software pointed to a greater concern of the players regarding visual aspects and non-functional requirements. As for hardware-related aspects, the biggest concern is related to limitations in mechanisms that are responsible for providing input data to applications.

[1]  Charles B. Owen,et al.  Review on cybersickness in applications and visual displays , 2016, Virtual Reality.

[2]  Claes Wohlin,et al.  Experimentation in Software Engineering , 2000, The Kluwer International Series in Software Engineering.

[3]  Yuri Antonio Gonçalves Vilas Boas.  Overview of Virtual Reality Technologies , 2012 .

[4]  Luciano Baresi,et al.  An Introduction to Software Testing , 2006, FoVMT.

[5]  Jeannie Novak,et al.  Game Development Essentials: Game QA & Testing , 2009 .

[6]  Hassan Reza,et al.  A Review on Usability and Performance Evaluation in Virtual Reality Systems , 2019, 2019 International Conference on Computational Science and Computational Intelligence (CSCI).

[7]  Vahid Balali,et al.  A Survey Study to Understand Industry Vision for Virtual and Augmented Reality Applications in Design and Construction , 2020, ArXiv.

[8]  Rafael Capilla,et al.  Software Architectures for Designing Virtual Reality Applications , 2004, EWSA.

[9]  Outi Sievi-Korte,et al.  Challenges and recommended practices for software architecting in global software development , 2019, Inf. Softw. Technol..

[10]  Jean Scholtz,et al.  Using consumer demands to bridge the gap between software engineering and usability engineering , 2003, Softw. Process. Improv. Pract..

[11]  K. Oakley,et al.  The Routledge companion to the cultural industries , 2015 .

[12]  Fumio Kishino,et al.  Augmented reality: a class of displays on the reality-virtuality continuum , 1995, Other Conferences.

[13]  Gerald Albaum,et al.  The Likert Scale Revisited , 1997 .

[14]  Richard H. Y. So,et al.  Effects of Navigation Speed on Motion Sickness Caused by an Immersive Virtual Environment , 2001, Hum. Factors.

[15]  Roger S. Pressman,et al.  Software Engineering: A Practitioner's Approach , 1982 .

[16]  Fátima L. S. Nunes,et al.  Towards the Systematic Testing of Virtual Reality Programs , 2019, 2019 21st Symposium on Virtual and Augmented Reality (SVR).

[17]  Adriano Bezerra,et al.  Definition of Test Criteria Based on the Scene Graph for VR Applications , 2011, 2011 XIII Symposium on Virtual Reality.

[18]  Steven M. LaValle,et al.  Virtual Reality , 2004, Heilberufe.

[19]  Martin Höst,et al.  Guidelines for Conducting Surveys in Software Engineering , 2015 .

[20]  Li,et al.  Software Testing Applications Based on a Virtual Reality System , 2007 .

[21]  Márcio Eduardo Delamaro,et al.  An automated functional testing approach for virtual reality applications , 2018, Softw. Test. Verification Reliab..

[22]  Carl Machover,et al.  Virtual reality , 1994, IEEE Computer Graphics and Applications.

[23]  Ferran Argelaguet,et al.  A survey of 3D object selection techniques for virtual environments , 2013, Comput. Graph..

[24]  Ulas Güleç,et al.  A Literature Survey: Is it Necessary to Develop a New Software Development Methodology for Virtual Reality Projects? , 2017, J. Univers. Comput. Sci..

[25]  Simon Davis,et al.  A Systematic Review of Cybersickness , 2014, IE.

[26]  Elena Marchiori,et al.  Analysis of users’ heart rate data and self-reported perceptions to understand effective virtual reality characteristics , 2018, J. Inf. Technol. Tour..