Recognizing Planned, Multiperson Action

Multiperson action recognition requires models of structured interaction between people and objects in the world. This paper demonstrates how highly structured, multiperson action can be recognized from noisy perceptual data using visually grounded goal-based primitives and low-order temporal relationships that are integrated in a probabilistic framework.The representation, which is motivated by work in model-based object recognition and probabilistic plan recognition, makes four principal assumptions: (1) the goals of individual agents are natural atomic representational units for specifying the temporal relationships between agents engaged in group activities, (2) a high-level description of temporal structure of the action using a small set of low-order temporal and logical constraints is adequate for representing the relationships between the agent goals for highly structured, multiagent action recognition, (3) Bayesian networks provide a suitable mechanism for integrating multiple sources of uncertain visual perceptual feature evidence, and (4) an automatically generated Bayesian network can be used to combine uncertain temporal information and compute the likelihood that a set of object trajectory data is a particular multiagent action.The recognition method is tested using a database of American football play descriptions and manually acquired but noisy player trajectories. The strengths and limitations of the system are discussed and compared with other multiagent recognition algorithms.

[1]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[2]  P Haddawy,et al.  Clinical simulation using context-sensitive temporal probability models. , 1995, Proceedings. Symposium on Computer Applications in Medical Care.

[3]  J. Azarewicz,et al.  Template-based multi-agent plan recognition for tactical situation assessment , 1989, [1989] Proceedings. The Fifth Conference on Artificial Intelligence Applications.

[4]  James F. Allen.  Towards a General Theory of Action and Time , 1984, Artif. Intell..

[5]  Tod S. Levitt,et al.  Model-Based Influence Diagrams for Machine Vision , 1989, UAI.

[6]  Edmund H. Durfee,et al.  Plan-based plan recognition models for the effective coordination of agents through observation , 1996 .

[7]  Michael P. Wellman,et al.  Accounting for Context in Plan Recognition, with Application to Traffic Monitoring , 1995, UAI.

[8]  Milind Tambe,et al.  Tracking Dynamic Team Activity , 1996, AAAI/IAAI, Vol. 1.

[9]  Edmund H. Durfee,et al.  The Automated Mapping of Plans for Plan Recognition , 1994, AAAI.

[10]  A F Bobick,et al.  Movement, activity and action: the role of knowledge in the perception of motion. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[11]  Arthur R. Pope.  Model-Based Object Recognition - A Survey of Recent Research , 1994 .

[12]  W. Eric L. Grimson,et al.  Localizing Overlapping Parts by Searching the Interpretation Tree , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Y. Shoham.  Reasoning About Change: Time and Causation from the Standpoint of Artificial Intelligence , 1987 .

[14]  Sarit Kraus,et al.  Collaborative Plans for Complex Group Action , 1996, Artif. Intell..

[15]  L. Zadeh.  Fuzzy sets as a basis for a theory of possibility , 1999 .

[16]  William Grimson,et al.  Object recognition by computer - the role of geometric constraints , 1991 .

[17]  Takeo Kanade,et al.  Visual Tracking of High DOF Articulated Structures: an Application to Human Hand Tracking , 1994, ECCV.

[18]  Hans-Hellmut Nagel,et al.  From image sequences towards conceptual descriptions , 1988, Image Vis. Comput..

[19]  Ashwin Ram,et al.  Needles in a Haystack: Plan Recognition in Large Spatial Domains Involving Multiple Agents , 1998, AAAI/IAAI.

[20]  Aaron F. Bobick,et al.  Closed-world tracking , 1995, Proceedings of IEEE International Conference on Computer Vision.

[21]  Stuart J. Russell,et al.  The BATmobile: Towards a Bayesian Automated Taxi , 1995, IJCAI.

[22]  Claudio S. Pinhanez,et al.  Representation and recognition of action in interactive spaces , 1999 .

[23]  Gudula Retz-Schmidt,et al.  A REPLAI of SOCCER: Recognizing Intentions in the Domain of Soccer Games , 1988, European Conference on Artificial Intelligence.

[24]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[25]  John McCarthy,et al.  SOME PHILOSOPHICAL PROBLEMS FROM THE STANDPOINT OF ARTI CIAL INTELLIGENCE , 1987 .

[26]  Michael I. Jordan,et al.  Probabilistic Independence Networks for Hidden Markov Probability Models , 1997, Neural Computation.

[27]  Eamonn J. Keogh,et al.  Learning augmented Bayesian classifiers: A comparison of distribution-based and classification-based approaches , 1999, AISTATS.

[28]  Gregory M. Provan,et al.  Dynamic Network Construction and Updating Techniques for the Diagnosis of Acute Abdominal Pain , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  B. Habibi,et al.  Pengi : An Implementation of A Theory of Activity , 1998 .

[30]  Dana H. Ballard,et al.  Computer Vision , 1982 .

[31]  Robert P. Goldman,et al.  Dynamic construction of belief networks , 1990, UAI.

[32]  Jitendra Malik,et al.  Automatic Symbolic Traffic Scene Analysis Using Belief Networks , 1994, AAAI.

[33]  Tieniu Tan,et al.  Agent orientated annotation in model based visual surveillance , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[34]  P. Schönemann.  On artificial intelligence , 1985, Behavioral and Brain Sciences.

[35]  Peter Haddawy,et al.  A Theoretical Framework for Context-Sensitive Temporal Probability Model Construction with Application to Plan Projection , 1995, UAI.

[36]  Henry A. Kautz,et al.  Generalized Plan Recognition , 1986, AAAI.

[37]  Philip R. Cohen,et al.  Plans as Complex Mental Attitudes , 2003 .

[38]  Y. Chien,et al.  Pattern classification and scene analysis , 1974 .

[39]  John S. Breese,et al.  CONSTRUCTION OF BELIEF AND DECISION NETWORKS , 1992, Comput. Intell..

[40]  H. Buxton,et al.  Advanced visual surveillance using Bayesian networks , 1997 .

[41]  Stephen S. Intille,et al.  Visual recognition of multiagent action , 1999 .

[42]  Aaron F. Bobick,et al.  Learning visual behavior for gesture analysis , 1995, Proceedings of International Symposium on Computer Vision - ISCV.

[43]  Samuel Holtzman,et al.  Intelligent decision systems , 1988 .

[44]  Robert P. Goldman,et al.  A Bayesian Model of Plan Recognition , 1993, Artif. Intell..

[45]  Christopher M. Brown,et al.  Where to Look Next Using a Bayes Net: Incorporating Geometric Relations , 1992, ECCV.