Semi-supervised learning of a POMDP model of Patient-Caregiver Interactions

This paper presents a decision theoretic model of interactions between assistive technology and users during activities of daily living. The model is a partially observable Markov decision process whose goal is to monitor a user, assist the user during each activity, maintain indicators of overall user health, and adapt to changes. The key idea behind the model is that it is relatively easy to specify, and can be applied to many activities of daily living with little modification. The key contribution of this paper is to show how such a model can be learned without knowing the classes of behaviors of the user a priori. This semi-supervised learning will enable assistive technologies to be applied ubiquitously for many different activities. We give some results from a preliminary version of the model for the task of handwashing.

[1]  Yoshua Bengio,et al.  Markovian Models for Sequential Data , 2004 .

[2]  Elizabeth D. Mynatt,et al.  Increasing the opportunities for aging in place , 2000, CUU '00.

[3]  Edward H. Adelson,et al.  Probability distributions of optical flow , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Nando de Freitas,et al.  Bayesian Feature Weighting for Unsupervised Learning, with Application to Object Recognition , 2003, AISTATS.

[5]  Milind Tambe,et al.  Exploiting belief bounds: practical POMDPs for personal assistant agents , 2005, AAMAS '05.

[6]  Henry A. Kautz,et al.  Learning and inferring transportation routines , 2004, Artif. Intell..

[7]  Joelle Pineau,et al.  Towards robotic assistants in nursing homes: Challenges and results , 2003, Robotics Auton. Syst..

[8]  W. Lovejoy.  A survey of algorithmic methods for partially observed Markov decision processes , 1991 .

[9]  Yan Huang,et al.  ARGMode - Activity Recognition using Graphical Models , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[10]  Jesse Hoey,et al.  A Decision-Theoretic Approach to Task Assistance for Persons with Dementia , 2005, IJCAI.

[11]  Karen Zita Haigh,et al.  Learning Models of Human Behaviour with Sequential Patterns , 2002 .

[12]  A Mihailidis.  The development of an intelligent cognitive orthosis to facilitate handwashing for persons with moderate-to-severe dementia. , 2002 .

[13]  Yoshua Bengio,et al.  Input-output HMMs for sequence processing , 1996, IEEE Trans. Neural Networks.

[14]  Roland T. Chin,et al.  On Image Analysis by the Methods of Moments , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Henry A. Kautz,et al.  An Overview of the Assisted Cognition Project , 2002 .

[16]  A. Mihailidis,et al.  Assistive technology for cognitive rehabilitation: State of the art , 2004 .

[17]  Svetha Venkatesh,et al.  Recognizing and monitoring high-level behaviors in complex spatial environments , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[18]  Alex Pentland,et al.  Coupled hidden Markov models for complex action recognition , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Jesse Hoey,et al.  Decision Theoretic Modeling of Human Facial Displays , 2004, ECCV.

[20]  Joelle Pineau,et al.  Point-based value iteration: An anytime algorithm for POMDPs , 2003, IJCAI.

[21]  M. Veloso,et al.  Using Sparse Visual Data to Model Human Activities in Meetings , 2004 .

[22]  Christoph Bregler,et al.  Learning and recognizing human dynamics in video sequences , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Jesse Hoey,et al.  Solving POMDPs with Continuous or Large Discrete Observation Spaces , 2005, IJCAI.

[24]  Martha E. Pollack,et al.  Autominder: an intelligent cognitive orthotic system for people with memory impairment , 2003, Robotics Auton. Syst..

[25]  Jesse Hoey,et al.  Value directed learning of gestures and facial displays , 2004, CVPR 2004.

[26]  Jesse Hoey,et al.  POMDP Models for Assistive Technology , 2005, AAAI Fall Symposium: Caring Machines.

[27]  Craig Boutilier,et al.  VDCBPI: an Approximate Scalable Algorithm for Large POMDPs , 2004, NIPS.

[28]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .