AQUArIUM - A suite of software measures for HCI quality evaluation of ubiquitous mobile applications

Five characteristics for HCI evaluation in ubiquitous applications are selected.Twenty-four quality measures to evaluate these five characteristics are proposed.A theoretical and empirical validations were performed. Ubiquitous computing has changed the way users interact with technology. Its applications are everywhere, supporting users in everyday activities in a transparent way with little or no need for attention. To ensure the adoption of these applications, it is essential to assess the quality of the interaction with its users. To do that, measurements can be applied to obtain data about quality characteristics in a software product. In a previous study about quality characteristics and software measures for ubiquitous computing, we have identified a gap in the literature regarding software measures for evaluating essential quality characteristics in ubiquitous systems. Therefore, this paper proposes a suite of well-defined software measures (twenty-four in total), called AQUArIUM, to evaluate human-computer interaction (HCI) in ubiquitous applications for mobile devices. These measures address five quality characteristics of ubiquitous computing: Context-awareness, Mobility, Attention, Calmness and Transparency. The proposed suite was validated theoretically and empirically. The results showed not only that the suite is feasible, but also that it indicates specific problems of HCI quality, which are helpful in improving the evaluated ubiquitous mobile application.

[1]  Shensheng Zhang,et al.  Adaptive Service Delivery for Mobile Users in Ubiquitous Computing Environments , 2006, UIC.

[2]  Luc Martens,et al.  Proposed Framework for Evaluating Quality of Experience in a Mobile, Testbed-oriented Living Lab Setting , 2010, Mob. Networks Appl..

[3]  Bachar El-Hassan,et al.  A Flexible Privacy and Trust Based Context-Aware Secure Framework , 2010, ICOST.

[4]  Jing Xu,et al.  A Pilot Study of an Inspection Framework for Automated Usability Guideline Reviews of Mobile Health Applications , 2014, Wireless Health.

[5]  Brian P. Bailey,et al.  Towards a pervasive computing benchmark , 2005, Third IEEE International Conference on Pervasive Computing and Communications Workshops.

[6]  Myung Hwan Yun,et al.  A User-Centered Approach for Ubiquitous Service Evaluation: An Evaluation Metrics Focused on Human-System Interaction Capability , 2008, APCHI.

[7]  Jesús Favela,et al.  Uncertainty Management in Context-Aware Applications: Increasing Usability and User Trust , 2011, Wirel. Pers. Commun..

[8]  David Garlan,et al.  Project Aura: Toward Distraction-Free Pervasive Computing , 2002, IEEE Pervasive Comput..

[9]  Yvonne Rogers,et al.  Interaction Design: Beyond Human-Computer Interaction , 2002 .

[10]  Ronan Fitzpatrick,et al.  Privacy metrics in ubiquitous computing applications , 2010, 2010 International Conference for Internet Technology and Secured Transactions.

[11]  Claes Wohlin,et al.  Experimentation in Software Engineering , 2000, The Kluwer International Series in Software Engineering.

[12]  José Maria Monteiro,et al.  Challenges for usability testing in ubiquitous systems , 2014, IHM.

[13]  Shari Lawrence Pfleeger,et al.  Software Metrics : A Rigorous and Practical Approach , 1998 .

[14]  Káthia Marçal de Oliveira,et al.  Quality characteristics and measures for human–computer interaction evaluation in ubiquitous systems , 2017, Software Quality Journal.

[15]  Markku Turunen,et al.  Evaluating the experiential user experience of public display applications in the wild , 2013, MUM.

[16]  Gavriel Salvendy,et al.  Number of people required for usability evaluation , 2010, Commun. ACM.

[17]  Henriette Cramer,et al.  Interacting with Adaptive Systems , 2010, Interactive Collaborative Information Systems.

[18]  Ubejd Shala,et al.  Indoor Positioning using Sensor-fusion in Android Devices , 2011 .

[19]  Watts S. Humphrey,et al.  Characterizing the software process: a maturity framework , 1988, IEEE Software.

[20]  William A. Florac,et al.  Goal-Driven Software Measurement. A Guidebook. , 1996 .

[21]  Chao Wang,et al.  Definitions of predictability for Cyber Physical Systems , 2016, J. Syst. Archit..

[22]  Rahat Iqbal,et al.  User-centred design and evaluation of ubiquitous services , 2005, SIGDOC '05.

[23]  Káthia Marçal de Oliveira,et al.  Evaluating an IoT Application Using Software Measures , 2017, HCI.

[24]  Andrew Sears,et al.  Human-Computer Interaction: Development Process , 2009 .

[25]  Mario Piattini,et al.  Software Product Quality Evaluation Using ISO/IEC 25000 , 2014, ERCIM News.

[26]  António Rafael Rodrigues Franco.  WiFi Roaming along urban routes , 2014 .

[27]  Kai Petersen,et al.  Worldviews, Research Methods, and their Relationship to Validity in Empirical Software Engineering Research , 2013, 2013 Joint Conference of the 23rd International Workshop on Software Measurement and the 8th International Conference on Software Process and Product Measurement.

[28]  Michael Collins,et al.  Evaluating Trust-Based Access Control for Social Interaction , 2009, 2009 Third International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies.

[29]  Jiannong Cao,et al.  Application mobility in pervasive computing: A survey , 2013, Pervasive Mob. Comput..

[30]  Cmmi Product Team.  CMMI for Development, Version 1.2 , 2010 .

[31]  王纬.  ISO／IEC 12207《信息技术—软件生存期过程简介》 , 2002 .

[32]  Eran Toch.  Super-Ego: a framework for privacy-sensitive bounded context-awareness , 2011, CASEMANS '11.

[33]  Egon Berghout,et al.  The Goal/Question/Metric method: a practical guide for quality improvement of software development , 1999 .

[34]  Didar Zowghi,et al.  An ontological framework to manage the relative conflicts between security and usability requirements , 2010, 2010 Third International Workshop on Managing Requirements Knowledge.

[35]  T. Devi,et al.  SOFTWARE METRICS VALIDATION METHODOLOGIES IN SOFTWARE ENGINEERING , 2014 .

[36]  B. Azvine,et al.  No Pervasive Computing without Intelligent Systems , 2004 .

[37]  Jadwiga Indulska,et al.  A survey of context modelling and reasoning techniques , 2010, Pervasive Mob. Comput..

[38]  Myung Hwan Yun,et al.  Usability assessment for ubiquitous services: Quantification of the interactivity in inter-personal services , 2012, 2012 IEEE International Conference on Management of Innovation & Technology (ICMIT).

[39]  Natalia Kryvinska,et al.  "Variated Availability" Approach to the Services Manageable Delivering , 2011, 2011 Fifth International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing.

[40]  Bruno Sousa,et al.  UEF: Ubiquity Evaluation Framework , 2011, WWIC.

[41]  James H. Aylor,et al.  Computer for the 21st Century , 1999, Computer.

[42]  P. Dourish,et al.  Context-Aware Computing , 2001 .

[43]  Mario Piattini,et al.  Using Technical-Action-Research to Validate a Framework for Authoring Software Engineering Methods , 2015, ICEIS.

[44]  Kurt Geihs,et al.  The user in the loop: Enabling user participation for self-adaptive applications , 2014, Future Gener. Comput. Syst..

[45]  Arkady B. Zaslavsky,et al.  Context Aware Computing for The Internet of Things: A Survey , 2013, IEEE Communications Surveys & Tutorials.

[46]  James R. Lewis,et al.  IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use , 1995, Int. J. Hum. Comput. Interact..

[47]  Jodi Forlizzi,et al.  Psycho-physiological measures for assessing cognitive load , 2010, UbiComp.

[48]  Ieee Standard,et al.  Adoption of ISO/IEC 15939:2007— Systems and Software Engineering— Measurement Process , 2009 .

[49]  Li-Chen Fu,et al.  Design and Realization of a Framework for Human–System Interaction in Smart Homes , 2012, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[50]  Sung-Bae Cho,et al.  Bayesian and behavior networks for context-adaptive user interface in a ubiquitous home environment , 2013, Expert Syst. Appl..

[51]  In-Young Ko,et al.  User-Centric Web Services for Ubiquitous Computing , 2010 .

[52]  James R. Lewis.  Computer System Usability Questionnaire , 2014 .

[53]  Kai Petersen,et al.  Systematic Mapping Studies in Software Engineering , 2008, EASE.

[54]  Alexander Egyed,et al.  Identifying requirements conflicts and cooperation: how quality attributes and automated traceability can help , 2004, IEEE Software.

[55]  Shari Lawrence Pfleeger,et al.  Software metrics (2nd ed.): a rigorous and practical approach , 1997 .

[56]  Jean Scholtz,et al.  Toward a Framework for Evaluating Ubiquitous Computing Applications , 2004, IEEE Pervasive Comput..

[57]  Michael Beigl,et al.  Using a Context Quality Measure for Improving Smart Appliances , 2007, 27th International Conference on Distributed Computing Systems Workshops (ICDCSW'07).

[58]  Ronald Poppe,et al.  Evaluating the Future of HCI: Challenges for the Evaluation of Emerging Applications , 2007, Artifical Intelligence for Human Computing.

[59]  Francoise Beaufays,et al.  “Your Word is my Command”: Google Search by Voice: A Case Study , 2010 .

[60]  Hausi A. Müller,et al.  A framework for evaluating quality-driven self-adaptive software systems , 2011, SEAMS '11.

[61]  George M. Giaglis,et al.  Delineating the Degree of 'Pervasiveness' in Pervasive Information Systems: An Assessment Framework and Design Implications , 2008, 2008 Panhellenic Conference on Informatics.

[62]  Thomas Skjødeberg Toftegaard,et al.  Requirements for an evaluation infrastructure for reliable pervasive healthcare research , 2012, 2012 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[63]  Manfred Broy,et al.  Formalizing the notion of adaptive system behavior , 2009, SAC '09.

[64]  Ronald Poppe,et al.  Evaluating the Future of HCI : Challenges for the Evaluation of Upcoming Applications , 2006 .

[65]  John Seely Brown,et al.  The coming age of calm technolgy , 1997 .

[66]  G. Y. Hong,et al.  A Research on Quality Assessment Technique for Ubiquitous Software and Middleware , 2004 .

[67]  David Garlan,et al.  Toward Distraction-Free Pervasive Computing , 2002 .

[68]  Joyce Statz.  Practical software measurement , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[69]  Gauthier Picard,et al.  Criteria for the evaluation of self-* systems , 2010, SEAMS '10.

[70]  S. Presser,et al.  Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording, and Context , 1996 .

[71]  Tao Sun,et al.  Performance Evaluation of Trust Management in Pervasive Computing , 2008, 22nd International Conference on Advanced Information Networking and Applications (aina 2008).

[72]  Gregory D. Abowd,et al.  What next, ubicomp?: celebrating an intellectual disappearing act , 2012, UbiComp.

[73]  Younghee Kim,et al.  A Quality Measurement Method of Context Information in Ubiquitous Environments , 2006, 2006 International Conference on Hybrid Information Technology.

[74]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[75]  Káthia Marçal de Oliveira,et al.  Integration of Subjective and Objective Usability Evaluation Based on IEC/IEC 15939: A Case Study for Traffic Supervision Systems , 2016, Int. J. Hum. Comput. Interact..

[76]  Sang-Hwan Kim,et al.  USABILITY CHALLENGES IN UBICOMP ENVIRONMENT , 2003 .

[77]  J.M.L. Rubio,et al.  Approach to a Quality Process for the Ubiquitous Software Development , 2007, Electronics, Robotics and Automotive Mechanics Conference (CERMA 2007).

[78]  Káthia Marçal de Oliveira,et al.  Using the GQM Method to Evaluate Calmness in Ubiquitous Applications , 2015, HCI.

[79]  Miroslaw Staron,et al.  Validating software measures using action research a method and industrial experiences , 2016, EASE.

[80]  Axel Küpper,et al.  Quality of Context: What It Is And Why We Need It , 2004 .

[81]  P. M. Clifford,et al.  The international vocabulary of basic and general terms in metrology , 1985 .

[82]  Didar Zowghi,et al.  Constructing a Catalogue of Conflicts among Non-functional Requirements , 2010, ENASE.

[83]  Gary Burnett,et al.  Evaluating the human-machine interface to vehicle navigation systems as an example of ubiquitous computing , 2001, Int. J. Hum. Comput. Stud..

[84]  Káthia Marçal de Oliveira,et al.  What Changes from Ubiquitous Computing to Internet of Things in Interaction Evaluation? , 2017, HCI.

[85]  Jong Kyu Choi,et al.  Usability Evaluation Framework for Ubiquitous Computing Device , 2008, 2008 Third International Conference on Convergence and Hybrid Information Technology.

[86]  Rogério de Lemos,et al.  Software Engineering for Self-Adaptive Systems [outcome of a Dagstuhl Seminar] , 2009, Software Engineering for Self-Adaptive Systems.

[87]  Guilherme Horta Travassos,et al.  Towards a framework to characterize ubiquitous software projects , 2012, Inf. Softw. Technol..

[88]  Mahadev Satyanarayanan,et al.  Pervasive computing: vision and challenges , 2001, IEEE Wirel. Commun..

[89]  Barry W. Boehm,et al.  Identifying Quality-Requirement Conflicts , 1996, IEEE Softw..

[90]  Jukka Riekki,et al.  Evaluating the Calmness of Ubiquitous Applications , 2004, PROFES.

[91]  Shari Lawrence Pfleeger,et al.  Towards a Framework for Software Measurement Validation , 1995, IEEE Trans. Software Eng..

[92]  Elizabeth Papadopoulou,et al.  A Privacy Framework for Personal Self-Improving Smart Spaces , 2009, 2009 International Conference on Computational Science and Engineering.

[93]  Yi-Hsing Chang,et al.  An Inquiry-based Ubiquitous Tour System , 2011, 2011 International Conference on Complex, Intelligent, and Software Intensive Systems.

[94]  Mark Weiser.  The computer for the 21st century , 1991 .

[95]  Said Ilias.  Function point counting practices manual , 2000 .

[96]  Mary Shaw,et al.  Software Engineering for Self-Adaptive Systems: A Research Roadmap , 2009, Software Engineering for Self-Adaptive Systems.

[97]  Brahim Chaib-draa,et al.  Repeated games for multiagent systems: a survey , 2013, The Knowledge Engineering Review.

[98]  Giovanni Acampora,et al.  Using ANFIS and FML for Deploying Transparent Services in Smart Environments , 2012, 2012 Sixth International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing.

[99]  Ulrich Amsel.  Product-Focused Software Process Improvement , 2011, Lecture Notes in Computer Science.

[100]  Andrea Vitaletti,et al.  Design and Initial Evaluation of a Ubiquitous Touch-Based Remote Grocery Shopping Process , 2009, 2009 First International Workshop on Near Field Communication.

[101]  Austin Henderson,et al.  Interaction design: beyond human-computer interaction , 2002, UBIQ.

[102]  Elizabeth A. Kemp,et al.  Interface evaluation for invisibility and ubiquity: an example from e-learning , 2008, CHINZ.

[103]  Jakob Nielsen,et al.  Estimating the number of subjects needed for a thinking aloud test , 1994, Int. J. Hum. Comput. Stud..

[104]  Schahram Dustdar,et al.  Quality of Context: models and applications for context-aware systems in pervasive environments , 2014, The Knowledge Engineering Review.