Intent Understanding Using an Activation Spreading Architecture

In this paper, we propose a new approach for recognizing intentions of humans by observing their activities with a color plus depth (RGB-D) camera. Activities and goals are modeled as a distributed network of inter-connected nodes in an Activation Spreading Network (ASN). Inspired by a formalism in hierarchical task networks, the structure of the network captures the hierarchical relationship between high-level goals and low-level activities that realize these goals. Our approach can detect intentions before they are realized and it can work in real-time. We also extend the formalism of ASNs to incorporate contextual information into intent recognition. We further augment the ASN formalism with special nodes and synaptic connections to model ordering constraints between actions, in order to represent and handle partial-order plans in our ASN. A fully functioning system is developed for experimental evaluation. We implemented a robotic system that uses our intent recognition to naturally interact with the user. Our ASN based intent recognizer is tested against three different scenarios involving everyday activities performed by a subject, and our results show that the proposed approach is able to detect low-level activities and recognize high-level intentions effectively in real-time. Further analysis shows that contextual and partial-order ASNs are able to discriminate between otherwise ambiguous goals.

[1]  Rodney A. Brooks,et al.  Humanoid robots , 2002, CACM.

[2]  Matthew Brand,et al.  Discovery and Segmentation of Activities in Video , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  David C. Hogg,et al.  Learning Variable-Length Markov Models of Behavior , 2001, Comput. Vis. Image Underst..

[4]  Eric Horvitz,et al.  Layered representations for human activity recognition , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[5]  Sven J. Dickinson,et al.  Recognize Human Activities from Partially Observed Videos , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Luís Moniz Pereira,et al.  State-of-the-art of intention recognition and its use in decision making , 2013, AI Commun..

[7]  Jon Williamson,et al.  Abduction, Reason, and Science: Processes of Discovery and Explanation , 2003 .

[8]  Jesse Hoey,et al.  Hierarchical unsupervised learning of facial expression categories , 2001, Proceedings IEEE Workshop on Detection and Recognition of Events in Video.

[9]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[10]  Manuela Veloso,et al.  Automated Robot Behavior Recognition , 2000 .

[11]  Gary Bradski,et al.  Computer Vision Face Tracking For Use in a Perceptual User Interface , 1998 .

[12]  Daniel S. Weld,et al.  UCPOP: A Sound, Complete, Partial Order Planner for ADL , 1992, KR.

[13]  日向 俊二 Kinect for Windowsアプリを作ろう , 2012 .

[14]  KimKyungnam,et al.  Real-time foreground-background segmentation using codebook model , 2005 .

[15]  Monica N. Nicolescu,et al.  Understanding human intentions via Hidden Markov Models in autonomous mobile robots , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[16]  James A. Hendler,et al.  HTN Planning: Complexity and Expressivity , 1994, AAAI.

[17]  Henry A. Kautz,et al.  Generalized Plan Recognition , 1986, AAAI.

[18]  Aaron F. Bobick,et al.  Recognition of Visual Activities and Interactions by Stochastic Parsing , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Monica N. Nicolescu,et al.  An Architecture for Understanding Intent Using a Novel Hidden Markov Formulation , 2008, Int. J. Humanoid Robotics.

[20]  Michael Harville,et al.  A Framework for High-Level Feedback to Adaptive, Per-Pixel, Mixture-of-Gaussian Background Models , 2002, ECCV.

[21]  Analía Amandi,et al.  Plan recognition for interface agents , 2007, Artificial Intelligence Review.

[22]  Diane J. Cook,et al.  Using Bayesian Networks for Daily Activity Prediction , 2013, AAAI Workshop: Plan, Activity, and Intent Recognition.

[23]  Frank Dignum,et al.  Mental State Abduction of BDI-Based Agents , 2008, DALT.

[24]  Anthony G. Cohn,et al.  Building qualitative event models automatically from visual input , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[25]  Aaron F. Bobick,et al.  Recognition and interpretation of parametric gesture , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[26]  Luís Moniz Pereira,et al.  Elder Care via Intention Recognition and Evolution Prospection , 2009, INAP.

[27]  Aaron F. Bobick,et al.  A Framework for Recognizing Multi-Agent Action from Visual Evidence , 1999, AAAI/IAAI.

[28]  Jake K. Aggarwal,et al.  Human activity recognition from 3D data: A review , 2014, Pattern Recognit. Lett..

[29]  Wolfgang Faber Answer Set Programming , 2013, Reasoning Web.

[30]  Stuart J. Russell,et al.  The BATmobile: Towards a Bayesian Automated Taxi , 1995, IJCAI.

[31]  Robert P. Goldman,et al.  Plan, Activity, and Intent Recognition: Theory and Practice , 2014 .

[32]  Bikramjit Banerjee,et al.  The complexity of multi-agent plan recognition , 2014, Autonomous Agents and Multi-Agent Systems.

[33]  Larry S. Davis,et al.  Real-time foreground-background segmentation using codebook model , 2005, Real Time Imaging.

[34]  Robert P. Goldman,et al.  A Bayesian Model of Plan Recognition , 1993, Artif. Intell..

[35]  J. van Leeuwen,et al.  Intelligent Agents V: Agents Theories, Architectures, and Languages , 2003, Lecture Notes in Computer Science.

[36]  J. Aggarwal,et al.  A Bayesian approach to human activity recognition , 1999, Proceedings Second IEEE Workshop on Visual Surveillance (VS'99) (Cat. No.98-89223).

[37]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[38]  Wolfgang Faber,et al.  Complexity of Model Checking and Bounded Predicate Arities for Non-ground Answer Set Programming , 2004, KR.

[39]  Mehdi Dastani,et al.  Programming mental state abduction , 2011, AAMAS.

[40]  Alex Pentland,et al.  Coupled hidden Markov models for complex action recognition , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[41]  Dana S. Nau,et al.  SHOP2: An HTN Planning System , 2003, J. Artif. Intell. Res..

[42]  Michael Wooldridge,et al.  The Belief-Desire-Intention Model of Agency , 1998, ATAL.

[43]  James A. Hendler,et al.  UMCP: A Sound and Complete Procedure for Hierarchical Task-network Planning , 1994, AIPS.

[44]  Sebastian Nowozin,et al.  Efficient Nonlinear Markov Models for Human Motion , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Pat Langley,et al.  Seeing Beyond Shadows: Incremental Abductive Reasoning for Plan Understanding , 2013, AAAI Workshop: Plan, Activity, and Intent Recognition.

[46]  Hema Swetha Koppula,et al.  Anticipating Human Activities Using Object Affordances for Reactive Robotic Response , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.