Using Cognitive Walkthrough and hybrid prototyping to gather user requirements in early design virtual reality prototypes

When it comes to user requirement gathering Virtual Reality (VR) projects investigate the usability of prototypes from the perspective of the end-users and their activities. To evaluate the usability of VR prototypes projects apply a variety of methods either single-perspective or hybrid methods. The literature suggests that hybrid methods produce more complete sets of requirements by highlighting both ‘in-world’ and user interface problems. The paper describes our experiences in using a single-perspective method that is a modified version of a Cognitive Walkthrough method for gathering user requirements in the REVERIE (Real and Virtual Engagement In Realistic Immersive Environment) project. We asked six experts to evaluate two hybrid VR prototypes with an educational context. As the software prototypes were in an early beta stage, we augmented them using storyboards and videos to simulate the missing user tasks. Also, we asked experts to get “into the shoes” of three archetypical users (represented as personas) while completing tasks with the prototypes. We found that our methodology was effective in highlighting a plethora of usability problems covering all aspects of the two prototypes. The identified problems were translated into 47 new requirements and prioritised based on the MoSCoW prioritization method. Most of the problems discovered were considered as essential in solving in the next design iteration of the software prototypes. Also, none of the problems discovered was considered as it was not a usability problem and hence, it should not be translated into a requirement. The performance of our methodology (single-perspective method and hybrid prototypes) was similar to hybrid methods. It also provided the foundation to develop a new method for the prioritisation of user requirements. Overall, we can conclude that using our methodology may provide a viable alternative to hybrid methods when it is required (e.g., when the expertise needed to use a hybrid method is not available). The paper concludes with a discussion of future work that focusses on integrating better training for experts and additional user groups in the evaluation process.