Design of an adaptive wayfinding system for individuals with cognitive impairments

Many individuals with cognitive impairments experience difficulty with wayfinding, that is, the process of traveling from one place to another. This functional limitation decreases their opportunity to live independently, maintain employment, and participate in their community. This dissertation addresses the design of a system to help such individuals in wayfinding by presenting personalized, multi-modal directions to users by mobile phone. By creating prototypes that present directions to individuals with cognitive impairments in real indoor and outdoor environments, I have observed a wide range of potential users' reactions to different types of directions, both turn-based directions common in current navigation aids and landmark-based directions that use photos augmented with path arrows. As the use of landmarks in wayfinding has been found to be beneficial, and landmark-based directions are becoming more widely available because of the growing availability of geotagged photos and other location information, my study results inform the creation of criteria for choosing landmarks useful to wayfinding. A recurring theme found in my studies has been the need to address the wide variation in individual wayfinding abilities and preferences. Addressing this need is a challenge as manually adjusting an interface's design, tailoring it for each individual user, has required significant cost in the past. To support such variation, I have developed a theoretical framework that creates customized and adaptable wayfinding directions to individual users. The framework relies on a Markov Decision Process (MDP) problem formulation to create understandable sequences of directions for a given user. The MDP uses a model to describe the user's likelihood of correctly following possible directions in different wayfinding situations. We can improve model accuracy by incorporating new usage data over time, allowing the system to adapt. Adaptation is an important capability as the physical and mental effort of wayfinding limits how much observational data can be collected at once. My user studies involving the framework generating directions show that individuals with cognitive impairments can successfully wayfind when they have access to tailored wayfinding directions, demonstrating the design's potential to reduce a significant barrier to their quality of living.

[1]  Steven E. Stock,et al.  Enhancing Independent Task Performance for Individuals with Mental Retardation through Use of a Handheld Self-Directed Visual and Audio Prompting System. , 2002 .

[2]  G E Lancioni,et al.  Promoting Independent Task Performance by Persons with Severe Developmental Disabilities through a New Computer-Aided System , 2000, Behavior modification.

[3]  A. Mihailidis,et al.  Assistive technology for cognitive rehabilitation: State of the art , 2004 .

[4]  J. Glover,et al.  Antecedents, descriptions and consequences of wandering in cognitively-impaired adults and the Safe Return (SR) program , 2001, American journal of Alzheimer's disease and other dementias.

[5]  Gaetano Borriello,et al.  Landmark-based pedestrian navigation from collections of geotagged photos , 2008, MUM '08.

[6]  Yang Li,et al.  Design Challenges and Principles for Wizard of Oz Testing of Location-Enhanced Applications , 2007, IEEE Pervasive Computing.

[7]  Wolfgang Wahlster,et al.  A resource-adaptive mobile navigation system , 2002, IUI '02.

[8]  Yusuf Ziya Özcan,et al.  Wayfinding in an Unfamiliar Environment , 2004 .

[9]  Alan J. Dix,et al.  Exploring the Effects of Target Location Size and Position System Accuracy on Location Based Applications , 2006, Pervasive.

[10]  Robert Laddaga,et al.  A location representation for generating descriptive walking directions , 2005, IUI.

[11]  Henry A. Kautz,et al.  Indoor wayfinding:: developing a functional interface for individuals with cognitive impairments , 2006, Assets '06.

[12]  Anind K. Dey,et al.  Navigate like a cabbie: probabilistic reasoning from observed context-aware behavior , 2008, UbiComp.

[13]  Kevin Lynch,et al.  The Image of the City , 1960 .

[14]  Henry A. Kautz,et al.  Customizing directions in an automated wayfinding system for individuals with cognitive impairment , 2009, Assets '09.

[15]  M. Sanders,et al.  Training Developmentally Disabled Adults in Independent Meal Preparation , 1989, Behavior modification.

[16]  Henry A. Kautz,et al.  Implications for Location Systems in Indoor Wayfinding for Individuals with Cognitive Impairments , 2006, 2006 Pervasive Health Conference and Workshops.

[17]  Jaewoo Chung,et al.  Going my way: a user-aware route planner , 2009, CHI.

[18]  Roberta L. Klatzky,et al.  Stated Preferences for Components of a Personal Guidance System for Nonvisual Navigation , 2004 .

[19]  Andrew J. May,et al.  Pedestrian navigation aids: information requirements and design implications , 2003, Personal and Ubiquitous Computing.

[20]  William Seager,et al.  Comparing physical, automatic and manual map rotation for pedestrian navigation , 2007, CHI.

[21]  Clayton Lewis Simplicity in cognitive assistive technology: a framework and agenda for research , 2006, Universal Access in the Information Society.

[22]  FischerGerhard User Modeling in HumanComputer Interaction , 2001 .

[23]  J. McGrath Methodology matters: doing research in the behavioral and social sciences , 1995 .

[24]  Dieter Fox,et al.  Gaussian Processes for Signal Strength-Based Location Estimation , 2006, Robotics: Science and Systems.

[25]  Krzysztof Z. Gajos,et al.  Improving the performance of motor-impaired users with automatically-generated, ability-based interfaces , 2008, CHI.

[26]  Karyn Moffatt 1 Generalizability in Research with Cognitively Impaired Individuals , 2006 .

[27]  Craig Zimring,et al.  Linking Objective Measures Of Space To Cognition And Action , 2003 .

[28]  Krzysztof Z. Gajos,et al.  Opportunity Knocks: A System to Provide Cognitive Assistance with Transportation Services , 2004, UbiComp.

[29]  Gaetano Borriello,et al.  Some sensor network elements for ubiquitous computing , 2005, IPSN 2005. Fourth International Symposium on Information Processing in Sensor Networks, 2005..

[30]  Joëlle Coutaz,et al.  A Wizard of Oz platform for the study of multimodal systems , 1993, INTERCHI Adjunct Proceedings.

[31]  Elizabeth D. Mynatt,et al.  Designing a cognitive aid for the home: a case-study approach , 2003, Assets '04.

[32]  Matthew Chalmers,et al.  EyeSpy: supporting navigation through play , 2009, CHI.

[33]  Wei-Hsun Chen MGuider: mobile guiding and tracking system in publictransit system for individuals with cognitive impairments , 2009, Assets '09.

[34]  James A. Landay,et al.  Personalizing routes , 2006, UIST.

[35]  F. Paas,et al.  Cognitive Load Theory and Instructional Design: Recent Developments , 2003 .

[36]  John C. Thomas,et al.  Minimizing ecological gaps in interface design , 1989, IEEE Software.

[37]  Gaetano Borriello,et al.  Harvesting of Location-Specific Information Through WiFi Networks , 2006, LoCA.

[38]  M. Rowe,et al.  People with dementia who become lost: Preventing injuries and death , 2003 .

[39]  M L Wehmeyer,et al.  National survey of the use of assistive technology by adults with mental retardation. , 1998, Mental retardation.

[40]  Emmanuel Munguia Tapia,et al.  Acquiring in situ training data for context-aware ubiquitous computing applications , 2004, CHI.

[41]  J. Epstein,et al.  Use of a Technological Prompting Device to Aid a Student with Attention Deficit Hyperactivity Disorder to Initiate and Complete Daily Tasks: An Exploratory Study , 2000 .

[42]  Luca Chittaro,et al.  Augmenting audio messages with visual directions in mobile guides: an evaluation of three approaches , 2005, Mobile HCI.

[43]  Stephen A. Brewster,et al.  Using Landmarks to Support Older People in Navigation , 2004, Mobile HCI.

[44]  Henry A. Kautz,et al.  Integrating Sensing and Cueing for More Effective Activity Reminders , 2008, AAAI Fall Symposium: AI in Eldercare: New Solutions to Old Problems.

[45]  Mike Wu,et al.  Participatory design of an orientation aid for amnesics , 2005, CHI.

[46]  Keith Cheverst,et al.  Developing a context-aware electronic tourist guide: some issues and experiences , 2000, CHI.

[47]  Christopher D. Wickens,et al.  An introduction to human factors engineering , 1997 .

[48]  M. Blades,et al.  The Cognition of Geographic Space , 2002 .

[49]  Robert Hardy,et al.  Design, implementation and evaluation of a novel public display for pedestrian navigation: the rotating compass , 2009, CHI.

[50]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[51]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[52]  Rainer Malaka,et al.  Evaluation of an Augmented Photograph-Based Pedestrian Navigation System , 2008, Smart Graphics.

[53]  Michael H. Bowling,et al.  Subjective Localization with Action Respecting Embedding , 2005, ISRR.

[54]  Hendrik Schulze,et al.  MEMOS: an interactive assistive system for prospective memory deficit compensation-architecture and functionality , 2004, Assets '04.

[55]  David A. Ross,et al.  Talking braille: a wireless ubiquitous computing network for orientation and wayfinding , 2005, Assets '05.

[56]  David A. Ross,et al.  Evaluation of orientation interfaces for wearable computers , 2000, Digest of Papers. Fourth International Symposium on Wearable Computers.

[57]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[58]  Rogerio DePaula,et al.  A Framework for the Adoption of Assistive Technology , 2002 .

[59]  Dieter Fox,et al.  Bayesian Filtering for Location Estimation , 2003, IEEE Pervasive Comput..

[60]  Jana Kosecka,et al.  Landmark-Based Pedestrian Navigation with Enhanced Spatial Reasoning , 2009, Pervasive.

[61]  Constantine Stephanidis,et al.  # 2001 Kluwer Academic Publishers. Printed in the Netherlands. Adaptive Techniques for Universal Access , 1999 .

[62]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[63]  Sunny Consolvo,et al.  Learning and Recognizing the Places We Go , 2005, UbiComp.

[64]  R. Golledge Wayfinding Behavior: Cognitive Mapping and Other Spatial Processes , 2010 .

[65]  Stephen Fickas,et al.  Investigating the usability of assistive user interfaces , 2003, Interact. Comput..

[66]  Elliot Cole,et al.  Computer-based cognitive prosthetics: assistive technology for the treatment of cognitive disabilities , 1998, Assets '98.

[67]  Adrian David Cheok,et al.  22nd International Conference on Human-Computer Interaction with Mobile Devices and Services , 2007, Lecture Notes in Computer Science.

[68]  S. Fickas,et al.  A profile of community navigation in adults with chronic cognitive impairments , 2005, Brain injury.

[69]  R Lubinski,et al.  Prevalence of speaking and hearing disabilities among adults with traumatic brain injury from a national household survey. , 1997, Brain injury.

[70]  Jana Kosecka,et al.  Visual Navigation for Mobile Devices , 2010, IEEE MultiMedia.

[71]  Gerhard Fischer,et al.  Socio-technical environments supporting people with cognitive disabilities using public transportation , 2005, TCHI.

[72]  Joelle Pineau,et al.  Pearl: A Mobile Robotic Assistant for the Elderly , 2002 .

[73]  Saila Ovaska,et al.  Usability evaluation involving participants with cognitive disabilities , 2004, NordiCHI '04.

[74]  Henry A. Kautz,et al.  Informing the design of an automated wayfinding system for individuals with cognitive impairments , 2009, 2009 3rd International Conference on Pervasive Computing Technologies for Healthcare.

[75]  Ben J. A. Kröse,et al.  Creating Design Guidelines for a Navigational Aid for Mild Demented Pedestrians , 2008, AmI.

[76]  Herre van Oostendorp,et al.  The paradox of the assisted user: guidance can be counterproductive , 2006, CHI.

[77]  Henry A. Kautz,et al.  Learning and inferring transportation routines , 2004, Artif. Intell..

[78]  Lucas Paletta,et al.  Visual object detection from mobile phone imagery for context awareness , 2005, Mobile HCI.

[79]  Doina Precup,et al.  Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning , 1999, Artif. Intell..

[80]  Gaetano Borriello,et al.  Location Systems for Ubiquitous Computing , 2001, Computer.

[81]  Melissa Dawe,et al.  Desperately seeking simplicity: how young adults with cognitive disabilities and their families adopt assistive technologies , 2006, CHI.

[82]  E. D. Newbigging,et al.  Therapy methodology. Riding the bus: teaching an adult with a brain injury to use a transit system to travel independently to and from work. , 1996, Brain injury.

[83]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[84]  Shih-Kai Tsai,et al.  WADER: a novel wayfinding system with deviation recovery for individuals with cognitive impairments , 2007, Assets '07.

[85]  Stephen Fickas,et al.  A pilot study exploring electronic (or e-mail) mail in users with acquired cognitive-linguistic impairments , 2003, Brain injury.

[86]  A C Repp,et al.  A comparison of static and dynamic presentation procedures on discrimination learning of individuals with severe or moderate mental retardation. , 1994, Research in developmental disabilities.

[87]  J. Schneider,et al.  Using talking lights to assist brain-injured patients with daily inpatient therapeutic schedule. , 2001, The Journal of head trauma rehabilitation.

[88]  Dieter Fox,et al.  Relational Object Maps for Mobile Robots , 2005, IJCAI.

[89]  Julie A. Jacko,et al.  A review and reappraisal of information technologies within a conceptual framework for individuals with disabilities , 2001, Universal Access in the Information Society.

[90]  Yao-Jen Chang,et al.  A context aware handheld wayfinding system for individuals with cognitive impairments , 2008, Assets '08.