Experience in Evaluating AAL Solutions in Living Labs

Ambient assisted living (AAL) is a complex field, where different technologies are integrated to offer solutions for the benefit of different stakeholders. Several evaluation techniques are commonly applied that tackle specific aspects of AAL; however, holistic evaluation approaches are lacking when addressing the needs of both developers and end-users. Living labs have been often used as real-life test and experimentation environments for co-designing AAL technologies and validating them with relevant stakeholders. During the last five years, we have been evaluating AAL systems and services in the framework of various research projects. This paper presents the lessons learned in this experience and proposes a set of harmonized guidelines to conduct evaluations in living labs.

[1]  Martha E. Pollack,et al.  Autominder: an intelligent cognitive orthotic system for people with memory impairment , 2003, Robotics Auton. Syst..

[2]  Sasitharan Balasubramaniam,et al.  Creating a distributed mobile networking testbed environment-through the Living Labs approach , 2006, 2nd International Conference on Testbeds and Research Infrastructures for the Development of Networks and Communities, 2006. TRIDENTCOM 2006..

[3]  Martin J. Russell,et al.  CogWatch - Automated Assistance and Rehabilitation of Stroke-Induced Action Disorders in the Home Environment , 2013, HCI.

[4]  Hong Sun,et al.  Promises and Challenges of Ambient Assisted Living Systems , 2009, 2009 Sixth International Conference on Information Technology: New Generations.

[5]  Filippo Cavallo,et al.  AALIANCE Ambient Assisted Living Roadmap , 2010, Ambient Intelligence and Smart Environments.

[6]  Charlotte Magnusson,et al.  CogWatch: Cognitive Rehabilitation for Apraxia and Action Disorganization Syndrome Patients , 2012 .

[7]  Wolfgang Straßer,et al.  Smart Camera Based Monitoring System and Its Application to Assisted Living , 2008, Proceedings of the IEEE.

[8]  Jakob Nielsen,et al.  Heuristic evaluation of user interfaces , 1990, CHI '90.

[9]  Jordi Aguiló,et al.  Domotic Hardware Infrastructure in PERSONA Project , 2010, ISAmI.

[10]  Erina Ferro,et al.  The PERSONA Service Platform for AAL Spaces , 2010, Handbook of Ambient Intelligence and Smart Environments.

[11]  Vicente Traver,et al.  Ambient assisted living spaces validation by services and devices simulation , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[12]  Jorge Cancela,et al.  State of the Art on Games for Health Focus on Parkinson's Disease Rehabilitation , 2014 .

[13]  Alex Mihailidis,et al.  A Survey on Ambient-Assisted Living Tools for Older Adults , 2013, IEEE Journal of Biomedical and Health Informatics.

[14]  Nicos Maglaveras,et al.  The REMOTE AAL Project: Remote Health and Social Care for Independent Living of Isolated Elderly with Chronic Conditions , 2011, HCI.

[15]  J. Summers,et al.  The pathogenesis of gait hypokinesia in Parkinson's disease. , 1994, Brain : a journal of neurology.

[16]  Wolfgang L. Zagler,et al.  Involvement of elderly citizens as potential end users of assistive technologies in the living lab Schwechat , 2009, PETRA '09.

[17]  Ann Ashburn,et al.  A meta‐analysis of six prospective studies of falling in Parkinson's disease , 2007, Movement disorders : official journal of the Movement Disorder Society.

[18]  Juan Bautista Mocholí,et al.  Virtual Reality for AAL Services Interaction Design and Evaluation , 2011, HCI.

[19]  Paolo Barsocchi,et al.  EvAAL, Evaluating AAL Systems through Competitive Benchmarking, the Experience of the 1st Competition , 2011, EvAAL.

[20]  Huiru Zheng,et al.  Visualization of data for ambient assisted living services , 2011, IEEE Communications Magazine.

[21]  Asbjørn Følstad,et al.  Living Labs for Innovation and Development of Information and Communication Technology: A Literature Review , 2008 .

[22]  Anna Ståhlbröst,et al.  Concept Design with a Living Lab Approach , 2009, 2009 42nd Hawaii International Conference on System Sciences.

[23]  Juan Carlos Augusto,et al.  Handbook of Ambient Intelligence and Smart Environments , 2009, HAIS 2010.

[24]  Luís Corte-Real,et al.  A New Living Lab for Usability Evaluation of ICT and Next Generation Networks for Elderly@Home , 2011, AAL.

[25]  H. S. Wolff,et al.  iRun: Horizontal and Vertical Shape of a Region-Based Graph Compression , 2022, Sensors.

[26]  Anna Ståhlbröst,et al.  Forming future IT: the living lab way of user involvement , 2008 .

[27]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[28]  María Teresa Arredondo,et al.  Design Framework for Ambient Assisted Living Platforms , 2009, HCI.

[29]  Valérie Gay,et al.  Using mobile phones to improve medication compliance and awareness for cardiac patients , 2010, PETRA '10.

[30]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[31]  Antonio Mora García,et al.  Deploying intelligent e-health services in a mobile gateway , 2013, Expert Syst. Appl..

[32]  Jeffrey M. Hausdorff,et al.  Falls and freezing of gait in Parkinson's disease: A review of two interconnected, episodic phenomena , 2004, Movement disorders : official journal of the Movement Disorder Society.

[33]  Martin R. Gibbs,et al.  The Magic Box and Collage: Responding to the challenge of distributed intergenerational play , 2009, Int. J. Hum. Comput. Stud..

[34]  Michael Hellenschmidt,et al.  Platforms for AAL Applications , 2010, EuroSSC.

[35]  Axel Helmer,et al.  A Service Oriented Platform for Health Services and Ambient Assisted Living , 2009, 2009 International Conference on Advanced Information Networking and Applications Workshops.

[36]  E. Katunina,et al.  [Epidemiology of Parkinson's disease]. , 2013, Zhurnal nevrologii i psikhiatrii imeni S.S. Korsakova.

[37]  Cathleen Wharton,et al.  Cognitive Walkthroughs: A Method for Theory-Based Evaluation of User Interfaces , 1992, Int. J. Man Mach. Stud..

[38]  M. Eriksson.  State-ofthe-art in utilizing Living Labs approach to user-centric ICT innovation-a European approach , 2005 .

[39]  Alain April,et al.  ISO/IEC SQuaRE. The second generation of standards for software product quality , 2003 .

[40]  Jesse Hoey,et al.  Zero Effort Technologies: Considerations, Challenges, and Use in Health, Wellness, and Rehabilitation , 2011, Synthesis Lectures on Assistive, Rehabilitative, and Health-Preserving Technologies.

[41]  Juan Carlos Augusto,et al.  Distributed Vision-Based Accident Management for Assisted Living , 2007, ICOST.

[42]  Luc Martens,et al.  Proposed Framework for Evaluating Quality of Experience in a Mobile, Testbed-oriented Living Lab Setting , 2010, Mob. Networks Appl..

[43]  Marc Hassenzahl,et al.  Hedonic, Emotional, and Experiential Perspectives on Product Quality , 2006 .

[44]  Stefano Chessa,et al.  Evaluating AAL Systems Through Competitive Benchmarking , 2012, Communications in Computer and Information Science.

[45]  Thomas Röfer,et al.  Mobility Assistance in the Bremen Ambient Assisted Living Lab , 2010 .

[46]  Elizabeth D. Mynatt,et al.  Digital family portraits: supporting peace of mind for extended family members , 2001, CHI.

[47]  Arie Reichman,et al.  The architecture of ambient assisted living system , 2011, 2011 IEEE International Conference on Microwaves, Communications, Antennas and Electronic Systems (COMCAS 2011).

[48]  Pablo García-Sánchez,et al.  Context-Awareness in a Service Oriented e-Health Platform , 2011, IWAAL.

[49]  Cui Jian,et al.  Speech and Gesture Interaction in an Ambient Assisted Living Lab , 2012, SMIAE@ACL.

[50]  Francesco Furfari,et al.  universAAL – An Open and Consolidated AAL Platform , 2011 .

[51]  Stefano Chessa,et al.  Evaluation of localization and activity recognition systems for ambient assisted living: The experience of the 2012 EvAAL competition , 2013, J. Ambient Intell. Smart Environ..

[52]  Tore Dybå,et al.  Building Theories in Software Engineering , 2008, Guide to Advanced Empirical Software Engineering.

[53]  Jeffrey M. Hausdorff,et al.  Falls in outpatients with Parkinson's disease: frequency, impact and identifying factors. , 2005, Journal of neurology.

[54]  J.M. Eklund,et al.  Information Technology for Assisted Living at Home: building a wireless infrastructure for assisted living , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[55]  Marc Hassenzahl,et al.  User experience - a research agenda , 2006, Behav. Inf. Technol..

[56]  Alan M. Wing,et al.  Remote Monitoring of Post-stroke Patients Suffering from Apraxia and Action Disorganisation Syndrome , 2013 .

[57]  Kavyashree Jamboti,et al.  Requirements and Evaluation of Safety Analysis Techniques for Ambient Assisted Living Systems , 2012, 2012 IEEE 23rd International Symposium on Software Reliability Engineering Workshops.

[58]  Matthias Struck,et al.  Feasibility of a wireless health monitoring system for prevention and health assessment of elderly people , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).