Challenges for usability testing in ubiquitous systems

Ubiquitous computing expands both the place where the software system is used and the traditional way of interacting with its users. This happens since technologies should be fully integrated in the user daily activities in such way they become indistinguishable. So, in this scenario, new characteristics can emerge like calmness, transparency and context awareness that should be evaluated to assure the ubiquitous system quality. One of the methods used for evaluation of traditional systems is usability testing. Then, in this paper, we identify the main challenges to perform usability testing in ubiquitous systems based on an overview of the literature studies and on our own experience in testing this kind of systems. We also describe our ongoing work where we investigate usability testing to be designed considering context-awareness and to be supported by specific measurements to the ubiquitous computing scenario.

[1]  Estefanía Martín,et al.  Accessible smartphones for blind users: A case study for a wayfinding system , 2014, Expert Syst. Appl..

[2]  Gary Burnett,et al.  Evaluating the human-machine interface to vehicle navigation systems as an example of ubiquitous computing , 2001, Int. J. Hum. Comput. Stud..

[3]  Jong Kyu Choi,et al.  Usability Evaluation Framework for Ubiquitous Computing Device , 2008, 2008 Third International Conference on Convergence and Hybrid Information Technology.

[4]  Jean Scholtz,et al.  Toward a Framework for Evaluating Ubiquitous Computing Applications , 2004, IEEE Pervasive Comput..

[5]  Brian P. Bailey,et al.  Towards a pervasive computing benchmark , 2005, Third IEEE International Conference on Pervasive Computing and Communications Workshops.

[6]  Sung-Bae Cho,et al.  Bayesian and behavior networks for context-adaptive user interface in a ubiquitous home environment , 2013, Expert Syst. Appl..

[7]  Christian Kray,et al.  Evaluating Pervasive and Ubiquitous Systems , 2008, IEEE Pervasive Computing.

[8]  Manfred Tscheligi,et al.  Towards a navigation system for blind people: a Wizard of Oz study , 2012, ASAC.

[9]  Sang-Hwan Kim,et al.  USABILITY CHALLENGES IN UBICOMP ENVIRONMENT , 2003 .

[10]  J.M.L. Rubio,et al.  Approach to a Quality Process for the Ubiquitous Software Development , 2007, Electronics, Robotics and Automotive Mechanics Conference (CERMA 2007).

[11]  Declan O'Sullivan,et al.  SimCon: A Tool to Support Rapid Evaluation of Smart Building Application Design using Context Simulation and Virtual Reality , 2010, J. Univers. Comput. Sci..

[12]  Rafael Jesús Segura,et al.  A scalable architecture for 3D map navigation on mobile devices , 2012, Personal and Ubiquitous Computing.

[13]  Paula Savioja,et al.  European Conference on Cognitive Ergonomics: Designing beyond the Product --- Understanding Activity and User Experience in Ubiquitous Environments , 2009 .

[14]  Stephen S. Yau,et al.  Testing context-sensitive middleware-based software applications , 2004, Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004..

[15]  Giovanni Acampora,et al.  Using ANFIS and FML for Deploying Transparent Services in Smart Environments , 2012, 2012 Sixth International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing.

[16]  Nikolaos M. Avouris,et al.  Considering Cost in Usability Evaluation of Mobile Applications: Who, Where and When , 2009, INTERACT.

[17]  Káthia Marçal de Oliveira,et al.  A Quality Model for Human-Computer Interaction Evaluation in Ubiquitous Systems , 2013, CLIHC.

[18]  Myung Hwan Yun,et al.  A User-Centered Approach for Ubiquitous Service Evaluation: An Evaluation Metrics Focused on Human-System Interaction Capability , 2008, APCHI.

[19]  Damianos Gavalas,et al.  Evaluation of a web recommender system in electronic and mobile tourism , 2012, Int. J. Web Eng. Technol..

[20]  Xingshe Zhou,et al.  Supporting rapid design and evaluation of pervasive applications: challenges and solutions , 2011, Personal and Ubiquitous Computing.

[21]  Myung Hwan Yun,et al.  Usability assessment for ubiquitous services: Quantification of the interactivity in inter-personal services , 2012, 2012 IEEE International Conference on Management of Innovation & Technology (ICMIT).

[22]  Rossana M. de Castro Andrade,et al.  LOCCAM - loosely coupled context acquisition middleware , 2013, SAC '13.

[23]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[24]  José Maria Monteiro,et al.  A Query Language for Data Access in Ubiquitous Environments , 2010, CLEI Electron. J..

[25]  Sirpa Riihiaho.  User testing when test tasks are not appropriate , 2009, ECCE.

[26]  Zdenek Míkovec,et al.  An evaluation tool for research of user behavior in a realistic mobile environment , 2011, Personal and Ubiquitous Computing.

[27]  Jakob Nielsen,et al.  Chapter 6 – Usability Testing , 1993 .

[28]  David S. Rosenblum,et al.  Automated Generation of Context-Aware Tests , 2007, 29th International Conference on Software Engineering (ICSE'07).

[29]  Dana Chisnell,et al.  Handbook of Usability Testing , 2009 .

[30]  Damianos Gavalas,et al.  A web-based pervasive recommendation system for mobile tourist guides , 2011, Personal and Ubiquitous Computing.