A virtual reality approach for usability assessment: case study on a wheelchair-mounted robot manipulator

The aim of the paper is to address an innovative methodology for assessing the usability of a product. This methodology is particularly suitable for designing products that provide their main functions through their control interfaces. In particular, this case study relates to the usability assessment of two control devices for a wheelchair-mounted robot manipulator to assist physically disabled people. The study focuses on defining a synthetic usability index on the basis of two currently used methods: the multi criteria decision analysis and the Saaty’s analytic hierarchy process. Several virtual reality (VR)-based experiments have been conducted, set up in accordance with a cross-array experimental plan, that adequately caters for both control and noise factors. Quantitative measures and subjective user evaluations have been collected to maximize the effectiveness, the efficiency and the satisfaction perceived by users while using the product. Compared to the literature on the subject, the proposed approach provides both more flexibility in defining quantitative indexes and more adequate results, even when involving only a small sample of users in the participatory design session. The use of VR technologies for the collection of the experimental data has been essential in terms of safety, costs and repeatability of the tests, as well as of the robustness with respect to noise factors.

[1]  H Das,et al.  Evaluation of a telerobotic system to assist surgeons in microsurgery. , 1999, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[2]  Jeff Sauro,et al.  A method to standardize usability metrics into a single score , 2005, CHI.

[3]  Roy Davies,et al.  Adapting Virtual Reality for the Participatory Design of Work Environments , 2004, Computer Supported Cooperative Work (CSCW).

[4]  Ergonomic requirements for office work with visual display terminals ( VDTs ) — Part 11 : Guidance on usability , 1998 .

[5]  Håkan Eftring,et al.  Technical results from Manus user trials , 1999 .

[6]  F. Zambetta,et al.  Improving the usability of an e-commerce web site through personalization , 2002 .

[7]  Pascual González,et al.  Usability-Oriented Quality Model Based on Ergonomic Criteria , 2008 .

[8]  Pasquale Erto,et al.  Statistics for Innovation , 2009 .

[9]  Helen Petrie,et al.  Remote usability evaluations With disabled people , 2006, CHI.

[10]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[11]  Francesco Caputo,et al.  VRTest : a Virtual Reality sysTEm for tranSporTation Design VRTest : a Virtual Reality sysTEm for tranSporTation design , 2007 .

[12]  Kasper Hornbæk,et al.  Measuring usability: are effectiveness, efficiency, and satisfaction really correlated? , 2000, CHI.

[13]  Oscar Mauricio Serrano Jaimes,et al.  EVALUACION DE LA USABILIDAD EN SITIOS WEB, BASADA EN EL ESTANDAR ISO 9241-11 (International Standard (1998) Ergonomic requirements For office work with visual display terminals (VDTs)-Parts II: Guidance on usability , 2012 .

[14]  Gerd Johansson,et al.  Adaptation of workplaces and homes for disabled people using computer-aided design , 1996 .

[15]  Sung H. Han,et al.  A methodology for developing a usability index of consumer electronic products , 2008 .

[16]  Rory A Cooper,et al.  Participatory design in the development of the wheelchair convoy system , 2008, Journal of NeuroEngineering and Rehabilitation.

[17]  Kari Kuutti,et al.  Virtual prototypes in usability testing , 2001, Proceedings of the 34th Annual Hawaii International Conference on System Sciences.

[18]  Nigel Bevan,et al.  Extending Quality in Use to Provide a Framework for Usability Measurement , 2009, HCI.

[19]  Marc Hassenzahl,et al.  MARC HASSENZAHL CHAPTER 3 The Thing and I: Understanding the Relationship Between User and Product , 2003 .

[20]  Zülal Güngör,et al.  The usability analysis with heuristic evaluation and analytic hierarchy process , 2009 .

[21]  Giuseppe Di Gironimo,et al.  Innovative assembly process for modular train and feasibility analysis in virtual environment , 2009 .

[22]  Umashankar Nagarajan,et al.  Human-Robot Physical Interaction with dynamically stable mobile robots , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[23]  Thomas L. Saaty,et al.  DECISION MAKING WITH THE ANALYTIC HIERARCHY PROCESS , 2008 .

[24]  Roy S. Kalawsky New methodologies and techniques for evaluating user performance in advanced 3D virtual interfaces , 1998 .

[25]  Harry Budi Santoso,et al.  Measuring the user experience , 2008 .

[26]  Kasper Hornbæk,et al.  Current practice in measuring usability: Challenges to usability studies and research , 2006, Int. J. Hum. Comput. Stud..

[27]  Erik Frekjmr,et al.  Measuring Usability: Are Effectiveness, Efficiency, and Satisfaction Really Correlated? , 2000 .

[28]  Adelaide Marzano,et al.  Human Robot Interaction in Virtual Reality , 2007, Eurographics Italian Chapter Conference.

[29]  Suat Tanaydin Robust Design and Analysis for Quality Engineering , 1996 .

[30]  Monique W. M. Jaspers,et al.  A comparison of usability methods for testing interactive health technologies: Methodological aspects and empirical evidence , 2009, Int. J. Medical Informatics.

[31]  Donald A. Norman,et al.  Emotional design , 2004, UBIQ.

[32]  Anna-Lisa Osvalder,et al.  Cognitive engineering methods as usability evaluation tools for medical equipment , 2004 .

[33]  Neff Walker,et al.  Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[34]  Giuseppe Di Gironimo,et al.  Designing in VR , 2009 .

[35]  Jürgen Sauer,et al.  The influence of user expertise and prototype fidelity in usability tests. , 2010, Applied ergonomics.

[36]  Riddhiman Ghosh,et al.  Discovering user profiles , 2009, WWW '09.

[37]  Bruno Siciliano,et al.  A Virtual-Reality-based Evaluation Environment for Wheelchair-mounted Manipulators , 2008, Eurographics Italian Chapter Conference.

[38]  Simon Richir,et al.  Wheelchair simulators: A review , 2009 .

[39]  Gerd Johansson,et al.  A suggested virtual reality methodology allowing people with cognitive disabilities to communicate their knowledge and experiences of public transport systems , 2008 .

[40]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[41]  Matthias Ehrgott,et al.  Multiple criteria decision analysis: state of the art surveys , 2005 .

[42]  Diyar Akay,et al.  Usability Ranking of Intercity Bus Passenger Seats Using Fuzzy Axiomatic Design Theory , 2006, CDVE.

[43]  James R. Simpson,et al.  Robust Design and Analysis for Quality Engineering , 1998 .

[44]  Brian Shackel,et al.  Human factors for informatics usability , 1991 .

[45]  Davy Monticolo,et al.  A collaborative Design for Usability approach supported by Virtual Reality and a Multi-Agent System embedded in a PLM environment , 2010, Comput. Aided Des..

[46]  Ming-Chyuan Lin,et al.  Using AHP and TOPSIS approaches in customer-driven product design process , 2008, Comput. Ind..

[47]  Akihiko Yokoyama,et al.  An Integrated Environment for Testing and Assessing the Usability of Information Appliances Using Digital and Physical Mock-Ups , 2007, HCI.