A Conversational Cognitive Aid with Activity Monitoring, Planning and Execution

We present recent work to add sensors and activity recognition to a commercially available cognitive aid, enabling context-aware activity monitoring, planning and cueing. The Conversational Assistant for Rehabilitation (CARE) system is a contextaware autonomous agent that interacts with users via spoken conversation, similar to user interaction with a human caregiver. It includes activity models for both the user and the virtual caregiver. CARE’s primary activity is talking with the user about their plan, performance and situation. CARE cannot directly execute user activities like bathing or eating, which are viewed as exogenous events by CARE. We begin with an overview of current executive function support provided by PEAT, the advanced cognitive aid that we are extending. We then discuss how we are extending PEAT with activity monitoring via sensors and activity recognition, a speech interface, and autonomous agent architecture with unified planning and execution.

[1]  Erann Gat,et al.  Experiences with an architecture for intelligent, reactive agents , 1997, J. Exp. Theor. Artif. Intell..

[2]  Richard Levinson,et al.  A General Programming Language for Unified Planning and Control , 1995, Artif. Intell..

[3]  Tara Estlin,et al.  Survey of Command Execution Systems for NASA Spacecraft and Robots , 2005 .

[4]  Richard Levinson,et al.  Human Frontal Lobes and AI Planning Systems , 1994, AIPS.

[5]  Henry A. Kautz,et al.  Fine-grained activity recognition by aggregating abstract object usage , 2005, Ninth IEEE International Symposium on Wearable Computers (ISWC'05).

[6]  Peter Norvig,et al.  A Unified Approach to Model-Based Planning and Execution , 2000 .

[7]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[8]  Context-Aware Computing,et al.  Inferring Activities from Interactions with Objects , 2004 .

[9]  Henry A. Kautz,et al.  Learning and inferring transportation routines , 2004, Artif. Intell..

[10]  Svetha Venkatesh,et al.  Activity recognition and abnormality detection with the switching hidden semi-Markov model , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Henry A. Kautz,et al.  Integrating Sensing and Cueing for More Effective Activity Reminders , 2008, AAAI Fall Symposium: AI in Eldercare: New Solutions to Old Problems.

[12]  Nicola Muscettola,et al.  IDEA: Planning at the Core of Autonomous Reactive Agents , 2002 .

[13]  R Levinson.  A Computer Model of Prefrontal Cortex Function , 1995, Annals of the New York Academy of Sciences.

[14]  Richard Levinson,et al.  Unified Planning and Execution for Autonomous Software Repair , 2005 .