How Robots Can Recognize Activities and Plans Using Topic Models

The ability to identify what humans are doing in the environment is a crucial element of successful responsive behavior in human-robot interaction. We examine new ways to perform plan recognition (PR) using natural language processing (NLP) techniques. PR often focuses on the structural relationships between consecutive observations and ordered activities that comprise plans. However, NLP commonly treats text as a bag-of-words, omitting such structural relationships and using topic models to break down the distribution of concepts discussed in documents. In this paper, we examine an analogous treatment of plans as distributions of activities. We explore the application of Latent Dirichlet Allocation topic models to human skeletal data of plan execution traces obtained from a RGB-D sensor. This investigation focuses on representing the data as text and interpreting learned activities as a form of activity recognition (AR). Additionally, we explain how the system may perform PR. The initial empirical results suggest that such NLP methods can be useful in complex PR and AR tasks.

[1]  Bart Selman,et al.  Unstructured human activity detection from RGBD images , 2011, 2012 IEEE International Conference on Robotics and Automation.

[2]  Greg Mori,et al.  IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL., NO. 1 Human Action Recognition by Semi-Latent Topic Models , 2022 .

[3]  Yoram Singer,et al.  The Hierarchical Hidden Markov Model: Analysis and Applications , 1998, Machine Learning.

[4]  Thomas L. Griffiths,et al.  Probabilistic Topic Models , 2007 .

[5]  Irfan A. Essa,et al.  Structure from Statistics - Unsupervised Activity Analysis using Suffix Trees , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[6]  Yang Wang,et al.  Human Action Recognition by Semilatent Topic Models , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[8]  Marc B. Vilain,et al.  Getting Serious about Parsing Plans : a Grammatical Analysis of Plan Recognition , 1990 .

[9]  Michael P. Wellman,et al.  Accounting for Context in Plan Recognition, with Application to Traffic Monitoring , 1995, UAI.

[10]  Bernt Schiele,et al.  Discovery of activity patterns using topic models , 2008 .

[11]  Gang Wang,et al.  Using Dependent Regions for Object Categorization in a Generative Framework , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[12]  Svetha Venkatesh,et al.  Learning Hierarchical Hidden Markov Models with General State Hierarchy , 2004, AAAI.

[13]  Rüdiger Dillmann,et al.  Feature Set Selection and Optimal Classifier for Human Activity Recognition , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[14]  Andrew McCallum,et al.  Polylingual Topic Models , 2009, EMNLP.

[15]  Mark Steedman,et al.  On Natural Language Processing and Plan Recognition , 2007, IJCAI.

[16]  Robert P. Goldman,et al.  Plan Recognition (Dagstuhl Seminar 11141) , 2011, Dagstuhl Reports.

[17]  Jiebo Luo,et al.  A Markov logic framework for recognizing complex events from multimodal data , 2013, ICMI '13.

[18]  Lynne E. Parker,et al.  4-dimensional local spatio-temporal features for human activity recognition , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.