Temporal and Object Relations in Plan and Activity Recognition for Robots Using Topic Models

Plan recognition (PR) and activity recognition (AR) systems are essential for effective human-robot interaction (HRI) since the robot needs to predict what other agents in the environment are doing (Lösch et al. 2007). Even when robots are designed to perform simple tasks such as lending an object to a person (Levine and Williams 2014), they cannot follow simple time-stamped commands. There is often considerable uncertainty about the way in which people operate and the duration of time they need to complete each action. Consequently, a robot that needs to perform complementary actions must be able to observe and understand what the person is doing, and the AR process must be executed in real time in order to respond to the current situation in a timely manner. Topic models such as Latent Dirichlet Allocation (LDA) (Blei, Ng, and Jordan 2003) have been used for this purpose (Sung et al. 2012; Zhang and Parker 2011). The goal of this work is twofold. First, we explore a new way of representing RGB-D sensor readings for use with LDA as an integration of both a PR and AR system. We follow this with a proposition of extensions of LDA that take temporal and/or object relational information into account during the recognition process.

[1]  Rüdiger Dillmann,et al.  Feature Set Selection and Optimal Classifier for Human Activity Recognition , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[2]  Eleanor J. Gibson,et al.  Perceiving the Affordances: A Portrait of Two Psychologists , 2001 .

[3]  Hanna M. Wallach,et al.  Topic modeling: beyond bag-of-words , 2006, ICML.

[4]  Yang Wang,et al.  Human Action Recognition by Semilatent Topic Models , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Cagatay Basdogan,et al.  The role of roles: Physical cooperation between humans and robots , 2012, Int. J. Robotics Res..

[6]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[7]  Raymond J. Mooney,et al.  Plan, Activity, and Intent Recognition: Theory and Practice , 2014 .

[8]  Tetsunari Inamura,et al.  Bayesian learning of tool affordances based on generalization of functional feature to estimate effects of unseen tools , 2013, Artificial Life and Robotics.

[9]  Max Welling,et al.  Fast collapsed gibbs sampling for latent dirichlet allocation , 2008, KDD.

[10]  Andrew McCallum,et al.  Topics over time: a non-Markov continuous-time model of topical trends , 2006, KDD '06.

[11]  Greg Mori,et al.  IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL., NO. 1 Human Action Recognition by Semi-Latent Topic Models , 2022 .

[12]  Bart Selman,et al.  Unstructured human activity detection from RGBD images , 2011, 2012 IEEE International Conference on Robotics and Automation.

[13]  Brian Charles Williams,et al.  Concurrent Plan Recognition and Execution for Human-Robot Teams , 2014, ICAPS.

[14]  Mark Steedman,et al.  On Natural Language Processing and Plan Recognition , 2007, IJCAI.

[15]  Bernt Schiele,et al.  Discovery of activity patterns using topic models , 2008 .

[16]  Thomas L. Griffiths,et al.  Integrating Topics and Syntax , 2004, NIPS.

[17]  Shlomo Zilberstein,et al.  Plan and Activity Recognition from a Topic Modeling Perspective , 2014, ICAPS.

[18]  Shichao Ou,et al.  Learning to Perceive Human Intention and Assist in A Situated Context , 2010 .

[19]  Lynne E. Parker,et al.  4-dimensional local spatio-temporal features for human activity recognition , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.