Obstacles and Perspectives for Evaluating Mixed Reality Usability

The goal of this paper is to survey the main issues with the ergonomic evaluation of MRS (Mixed Reality Systems) and to stimulate discussions for future research. A first point concerns definitions and specificities of MRS within the « reality / virtuality » continuum, and the incorporation of user issues. Another point concerns the combinatory character of the ergonomic knowledge to be applied to MRS entities (reality, « GUIs », « VR », and « MR » specific). A major issue concerns ergonomic evaluation methods, their current state, their advantages and drawbacks, particularly for user testing. Finally, the discussion points at various items which may be part of a future research agenda, such as the need for more usability data, for generic and well controlled experiments; for common testing platforms, for shared recommendations data bases, for design and assessment of inspection methods, for common task taxonomies and for common models of MR entities and situations, etc.; all of this possibly leading to increased knowledge based on shared benchmarks.