A framework for evaluating Ambient Assisted Living technologies and the experience of the universAAL project

Ambient Assisted Living (AAL) technologies aim at supporting elderly and disabled people in their environment, with affordable, easy to use and meaningful ICT tools. Given the complexity of AAL, software platforms are needed to abstract and reuse typical functionalities. To this purpose, the universAAL project was established to create an open platform that provides a standardized approach making it technically feasible and economically viable to develop AAL solutions. In order to make AAL technologies widely accepted, high quality must be guaranteed, but measuring quality in AAL is difficult because of the variety of stakeholders, the wide set of needs addressed and the heterogeneity of the employed technologies. This paper proposes a methodological framework for evaluating AAL solutions with the aim of prioritizing evaluation needs, customizing assessment techniques and producing value for the project. The framework is based on empirical methods and on scientific-like theory building mixed with the standard ISO/IEC 25000 for software quality. It can be applied for both technical and non-technical (end-user) evaluations. To support the framework, the results of its application in universAAL are shown, i.e., how it was used at different stages of the project for assessing the results, for taking decisions and for building scientific knowledge.

[1]  Dario Salvi,et al.  Key Technological Success Features for a Domain Specific Open Software Ecosystem for Ambient Assisted Living , 2012, IICS.

[2]  Klaus Marius Hansen,et al.  An empirical study of software architectures' effect on product quality , 2011, J. Syst. Softw..

[3]  Michael Hellenschmidt,et al.  A Modelling Framework for Ambient Assisted Living Validation , 2009, HCI.

[4]  Miguel J. Hornos,et al.  Enabling correct design and formal analysis of Ambient Assisted Living systems , 2012, J. Syst. Softw..

[5]  Fumio Akiyama,et al.  An Example of Software System Debugging , 1971, IFIP Congress.

[6]  Verena Fuchsberger,et al.  Ambient assisted living: elderly people's needs and how to face them , 2008, SAME '08.

[7]  Vicente Traver,et al.  Ambient assisted living spaces validation by services and devices simulation , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[8]  Ben J. A. Kröse,et al.  Ambient Monitoring from an Elderly-Centred Design Perspective: What, Who and How , 2011, AmI.

[9]  Julie Brown,et al.  The "Acceptance" of Ambient Assisted Living: Developing an Alternate Methodology to This Limited Research Lens , 2011, ICOST.

[10]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[11]  Paolo Ciancarini,et al.  Performance Analysis of Software Architectures via a Process Algebraic Description Language , 1999 .

[12]  Christopher C. Mayer,et al.  AALuis, a User Interface Layer That Brings Device Independence to Users of AAL Systems , 2012, ICCHP.

[13]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[14]  Andrew Glover,et al.  Continuous Integration: Improving Software Quality and Reducing Risk (The Addison-Wesley Signature Series) , 2007 .

[15]  Tibor Gyimóthy,et al.  Empirical validation of object-oriented metrics on open source software for fault prediction , 2005, IEEE Transactions on Software Engineering.

[16]  Bernadette Szajna,et al.  Empirical evaluation of the revised technology acceptance model , 1996 .

[17]  Tore Dybå,et al.  Building Theories in Software Engineering , 2008, Guide to Advanced Empirical Software Engineering.

[18]  Dave Astels,et al.  The RSpec Book: Behaviour Driven Development with RSpec, Cucumber, and Friends , 2010 .

[19]  Teresita Rojas,et al.  Construction of a Systemic Quality Model for Evaluating a Software Product , 2003, Software Quality Journal.

[20]  Alan R. Hevner,et al.  The Use of Focus Groups in Design Science Research , 2010 .

[21]  C. N. Payne Using composition and refinement to support security architecture trade-off analysis , 1999 .

[22]  Christian Nøhr,et al.  Statement on Reporting of Evaluation Studies in Health Informatics , 2007 .

[23]  Maurice H. Halstead,et al.  Elements of software science , 1977 .

[24]  David M. Weiss,et al.  Architecture reviews: practice and experience , 2005, IEEE Software.

[25]  Jan Bosch,et al.  Scenario-based software architecture reengineering , 1998, Proceedings. Fifth International Conference on Software Reuse (Cat. No.98TB100203).

[26]  Jan Bosch,et al.  Architecture-level modifiability analysis (ALMA) , 2004, J. Syst. Softw..

[27]  Rich Hilliard,et al.  Updating IEEE 1471: Architecture Frameworks and Other Topics , 2008, Seventh Working IEEE/IFIP Conference on Software Architecture (WICSA 2008).

[28]  M. Jørgensen Software quality measurement , 1999 .

[29]  Joseph P. Cavano,et al.  A framework for the measurement of software quality , 1978 .

[30]  Uwe Fachinger,et al.  Better Health and Ambient Assisted Living (AAL) from a global, regional and local economic perspective , 2010 .

[31]  Mary Beth Chrissis,et al.  CMMI for Development: Guidelines for Process Integration and Product Improvement , 2011 .

[32]  Paul Müller,et al.  Ambient Intelligence in Assisted Living: Enable Elderly People to Handle Future Interfaces , 2007, HCI.

[33]  Laura Lehtola,et al.  The Focus Group Method as an Empirical Tool in Software Engineering , 2008, Guide to Advanced Empirical Software Engineering.

[34]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[35]  Richard A. Krueger,et al.  Focus groups : a practical guide for applied research / by Richard A. Krueger , 1989 .

[36]  Andreas Braun,et al.  Ambient Assisted Living , 2014 .

[37]  Juan Carlos Augusto,et al.  Software simulation and verification to increase the reliability of Intelligent Environments , 2013, Adv. Eng. Softw..

[38]  Marc Hassenzahl,et al.  Hedonic, Emotional, and Experiential Perspectives on Product Quality , 2006 .

[39]  Juan Bautista Mocholí,et al.  Ontology for Modeling Interaction in Ambient Assisted Living Environments , 2010 .

[40]  Barry W. Boehm,et al.  Software Engineering Economics , 1993, IEEE Transactions on Software Engineering.

[41]  A. Williams EuroQol : a new facility for the measurement of health-related quality of life , 1990 .

[42]  James T. Miller,et al.  An Empirical Evaluation of the System Usability Scale , 2008, Int. J. Hum. Comput. Interact..

[43]  A. N. Oppenheim,et al.  Questionnaire Design, Interviewing and Attitude Measurement , 1992 .

[44]  Francesco Furfari,et al.  universAAL – An Open and Consolidated AAL Platform , 2011 .

[45]  C. Shapiro,et al.  Technology Adoption in the Presence of Network Externalities , 1986, Journal of Political Economy.

[46]  C. Banzato,et al.  Construct representation and definitions in psychopathology: the case of delusion , 2010, Philosophy, ethics, and humanities in medicine : PEHM.

[47]  Nicolette de Keizer,et al.  STARE-HI -Statement on Reporting of Evaluation Studies in Health Informatics , 2009, Yearbook of Medical Informatics.

[48]  Francesco Furfari,et al.  universAAL: Provisioning Platform for AAL Services , 2013, ISAmI.

[49]  Eila Niemelä,et al.  A Survey on Software Architecture Analysis Methods , 2002, IEEE Trans. Software Eng..

[50]  Leonard J. Bass,et al.  SAAM: a method for analyzing the properties of software architectures , 1994, Proceedings of 16th International Conference on Software Engineering.

[51]  Paolo Ciancarini,et al.  ÆMPA: a process algebraic description language for the performance analysis of software architectures , 2000, WOSP '00.

[52]  Cemal Yilmaz,et al.  Software Metrics , 2008, Wiley Encyclopedia of Computer Science and Engineering.

[53]  Jakob Nielsen,et al.  Usability inspection methods , 1994, CHI 95 Conference Companion.

[54]  Albert Westergren,et al.  The Experiences of Elderly People Living at Home Related to Their Receiving Meals Distributed by a Municipality in Sweden , 2012 .

[55]  Marjo Rauhala,et al.  Independent living, technology and ethics , 2003 .

[56]  Herbert Plischke,et al.  Ambient Assistive Technologies (AAT): socio-technology as a powerful tool for facing the inevitable sociodemographic challenges? , 2010, Philosophy, ethics, and humanities in medicine : PEHM.

[57]  Marita Franzke,et al.  Usability evaluation with the cognitive walkthrough , 1995, CHI 95 Conference Companion.

[58]  Alain April,et al.  ISO/IEC SQuaRE. The second generation of standards for software product quality , 2003 .

[59]  T. Peters,et al.  Assessing quality of life among British older people using the ICEPOP CAPability (ICECAP-O) measure , 2011, Applied health economics and health policy.

[60]  Marco Torchiano,et al.  On the effectiveness of the test-first approach to programming , 2005, IEEE Transactions on Software Engineering.

[61]  C. Oppenauer Motivation and needs for technology use in old age , 2009 .

[62]  Dave Astels,et al.  The RSpec Book , 2012 .

[63]  Adam C. Marshall,et al.  A relationship between software coverage metrics and reliability , 1994, Softw. Test. Verification Reliab..

[64]  William A. Florac Software Quality Measurement: A Framework for Counting Problems and Defects , 1992 .

[65]  PerOlof Bengtsson,et al.  Architecture-Level Modifiability Analysis , 2002 .

[66]  Noëlle Carbonell,et al.  Ambient multimodality: towards advancing computer accessibility and assisted living , 2006, Universal Access in the Information Society.

[67]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[68]  Stefano Chessa,et al.  Evaluation of localization and activity recognition systems for ambient assisted living: The experience of the 2012 EvAAL competition , 2013, J. Ambient Intell. Smart Environ..

[69]  Mickael Bech,et al.  A MODEL FOR ASSESSMENT OF TELEMEDICINE APPLICATIONS: MAST , 2012, International Journal of Technology Assessment in Health Care.

[70]  Jakob Nielsen,et al.  Chapter 4 – The Usability Engineering Lifecycle , 1993 .

[71]  Reiner Wichert,et al.  Can the Market Breakthrough in AAL be Achieved by a Large Scale Pilot , 2014 .

[72]  Joseph F. Dumas,et al.  A Practical Guide to Usability Testing , 1993 .

[73]  Julie Beadle-Brown,et al.  Quality of life outcomes for residents and quality ratings of care homes: is there a relationship? , 2012, Age and ageing.

[74]  Michael Scriven,et al.  An Introduction to Meta-Evaluation. , 1969 .

[75]  Francesco Furfari,et al.  universAAL: an Open Platform and Reference Specification for Building AAL Systems , 2011, ERCIM News.

[76]  Daniela E. Damian,et al.  Selecting Empirical Methods for Software Engineering Research , 2008, Guide to Advanced Empirical Software Engineering.

[77]  Roger S. Pressman,et al.  Software Engineering: A Practitioner's Approach , 1982 .

[78]  Giuseppe Turchetti,et al.  Health Technology Assessment , 2010, IEEE Engineering in Medicine and Biology Magazine.

[79]  Martina Ziefle,et al.  Smart Home Technologies: Insights into Generation-Specific Acceptance Motives , 2009, USAB.

[80]  Reiner Wichert,et al.  Ambient Assisted Living : 4. AAL-Kongress 2011 , 2011 .

[81]  D. K. Hammer,et al.  Scenario-based software architecture evaluation methods : an overview , 2002 .

[82]  Mikael Lindvall,et al.  Evaluating software architectures , 2004, Adv. Comput..