POMDP Based Action Planning and Human Error Detection

This paper presents a Partially Observable Markov Decision Process (POMDP) model for action planning and human errors detection, during Activities of Daily Living (ADLs). This model is integrated into a sub-component of an assistive system designed for stroke survivors; it is called the Artificial Intelligent Planning System (AIPS). Its main goal is to monitor the user’s history of actions during a specific task, and to provide meaningful assistance when an error is detected in his/her sequence of actions. To do so, the AIPS must cope with the ambiguity in the outputs of the other system’s components. In this paper, we first give an overview of the global assistive system where the AIPS is implemented, and explain how it interacts with the user to guide him/her during tea-making. We then define the POMDP models and the Monte Carlo Algorithm used to learn how to retrieve optimal prompts, and detect human errors under uncertainty.

[1]  José M. Cogollor,et al.  Using Human-Computer Interface for Rehabilitation of Activities of Daily Living (ADL) in Stroke Patients: Lessons from the First Prototype , 2014 .

[2]  Karl Johan Åström,et al.  Optimal control of Markov processes with incomplete state information , 1965 .

[3]  Jesse Hoey,et al.  Assisting persons with dementia during handwashing using a partially observable Markov decision process. , 2007, ICVS 2007.

[4]  C. T. Ng,et al.  Measures of distance between probability distributions , 1989 .

[5]  Pia Rotshtein,et al.  CogWatch: Intelligent agent-based system to assist stroke survivors during tea-making , 2015, 2015 SAI Intelligent Systems Conference (IntelliSys).

[6]  K. Pearson.  NOTES ON THE HISTORY OF CORRELATION , 1920 .

[7]  Roberto Pieraccini,et al.  A stochastic model of human-machine interaction for learning dialog strategies , 2000, IEEE Trans. Speech Audio Process..

[8]  A. Mihailidis,et al.  The COACH prompting system to assist older adults with dementia through handwashing: An efficacy study , 2008, BMC geriatrics.

[9]  Elena Deza,et al.  Encyclopedia of Distances , 2014 .

[10]  Martin J. Russell,et al.  Intelligent Assistive System Using Real-Time Action Recognition for Stroke Survivors , 2014, 2014 IEEE International Conference on Healthcare Informatics.

[11]  Sven Wachsmuth,et al.  TEBRA: An Automatic Prompting System for Persons with Cognitive Disabilities in Brushing Teeth , 2013, HEALTHINF.

[12]  Oliver Lemon,et al.  Cluster-based user simulations for learning dialogue strategies , 2006, INTERSPEECH.

[13]  M. Schwartz,et al.  Errorless learning in cognitive rehabilitation: A critical review , 2012, Neuropsychological rehabilitation.

[14]  Alison L Gibbs,et al.  On Choosing and Bounding Probability Metrics , 2002, math/0209021.

[15]  Kallirroi Georgila,et al.  Learning user simulations for information state update dialogue systems , 2005, INTERSPEECH.

[16]  Martha E. Pollack,et al.  Adaptive cognitive orthotics: combining reinforcement learning and constraint-based temporal reasoning , 2004, ICML.

[17]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..

[18]  Pascal Poupart,et al.  Factored partially observable Markov decision processes for dialogue management , 2005 .

[19]  Steve J. Young,et al.  Bayesian update of dialogue state: A POMDP framework for spoken dialogue systems , 2010, Comput. Speech Lang..

[20]  Jesse Hoey,et al.  A planning system based on Markov decision processes to guide people with dementia through activities of daily living , 2006, IEEE Transactions on Information Technology in Biomedicine.

[21]  Steve J. Young,et al.  A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies , 2006, The Knowledge Engineering Review.

[22]  Milica Gasic,et al.  The Hidden Information State model: A practical framework for POMDP-based spoken dialogue management , 2010, Comput. Speech Lang..

[23]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..