Analysis of multi-agent activity using petri nets

This paper presents the use of place/transition petri nets (PNs) for the recognition and evaluation of complex multi-agent activities. The PNs were built automatically from the activity templates that are routinely used by experts to encode domain-specific knowledge. The PNs were built in such a way that they encoded the complex temporal relations between the individual activity actions. We extended the original PN formalism to handle the propagation of evidence using net tokens. The evaluation of the spatial and temporal properties of the actions was carried out using trajectory-based action detectors and probabilistic models of the action durations. The presented approach was evaluated using several examples of real basketball activities. The obtained experimental results suggest that this approach can be used to determine the type of activity that a team has performed as well as the stage at which the activity ended.

[1]  Marta Bon,et al.  Errors and Mistakes in Automated Player Tracking , 2001 .

[2]  Matej Kristan,et al.  A trajectory-based analysis of coordinated team activity in a basketball game , 2009, Comput. Vis. Image Underst..

[3]  Larry S. Davis,et al.  Representation and Recognition of Events in Surveillance Video Using Petri Nets , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[4]  Irfan A. Essa,et al.  Learning Temporal Sequence Model from Partially Labeled Data , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[5]  Vinod Nair,et al.  Automated Visual Surveillance Using Hidden Markov Models , 2002 .

[6]  J. Pers,et al.  Physics-based modelling of human motion using Kalman filter and collision avoidance algorithm , 2005, ISPA 2005. Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, 2005..

[7]  John Kresse,et al.  The Complete Book of Man-to-Man Offense , 1998 .

[8]  Matej Kristan,et al.  Analysis of Player Motion in Sport Matches , 2008, Computer Science in Sport - Mission and Methods.

[9]  Thomas B. Moeslund,et al.  A Survey of Computer Vision-Based Human Motion Capture , 2001, Comput. Vis. Image Underst..

[10]  Matej Kristan,et al.  Automatic Evaluation of Organized Basketball Activity using Bayesian Networks , 2007 .

[11]  David C. Hogg,et al.  Representation and synthesis of behaviour using Gaussian mixtures , 2002, Image Vis. Comput..

[12]  Jake K. Aggarwal,et al.  Human motion: modeling and recognition of actions and interactions , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[13]  Peter J. Haas,et al.  Stochastic Petri Nets: Modelling, Stability, Simulation , 2002 .

[14]  Mubarak Shah,et al.  Motion-based recognition a survey , 1995, Image Vis. Comput..

[15]  Yan Huang,et al.  Propagation networks for recognition of partially ordered sequential action , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[16]  Aaron F. Bobick,et al.  Recognizing Planned, Multiperson Action , 2001, Comput. Vis. Image Underst..

[17]  Svetha Venkatesh,et al.  Activity recognition and abnormality detection with the switching hidden semi-Markov model , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[18]  Stanislav Kovacic,et al.  Observation and analysis of large-scale human motion. , 2002, Human movement science.

[19]  Robert J. Woodham,et al.  Analysis of player actions in selected hockey game situations , 2005, The 2nd Canadian Conference on Computer and Robot Vision (CRV'05).

[20]  Pascal Fua,et al.  Modeling people: Vision-based understanding of a person's shape, appearance, movement, and behaviour , 2006, Comput. Vis. Image Underst..

[21]  Samy Bengio,et al.  Automatic analysis of multimodal group actions in meetings , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Ehud Rivlin,et al.  Building Petri Nets from Video Event Ontologies , 2007, ISVC.

[23]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[24]  Kurt Jensen,et al.  Coloured Petri Nets , 1997, Monographs in Theoretical Computer Science An EATCS Series.

[25]  Marco Ajmone Marsan,et al.  Modelling with Generalized Stochastic Petri Nets , 1995, PERV.

[26]  Aaron F. Bobick,et al.  Action recognition using probabilistic parsing , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[27]  Stanislav Kovacic,et al.  Tracking People in Sport: Making Use of Partially Controlled Environment , 2001, CAIP.

[28]  Youtian Du,et al.  Human Interaction Representation and Recognition Through Motion Decomposition , 2007, IEEE Signal Processing Letters.

[29]  Yunqian Ma,et al.  Activity Recognition using Dynamic Bayesian Networks with Automatic State Selection , 2007, 2007 IEEE Workshop on Motion and Video Computing (WMVC'07).

[30]  Matej Kristan,et al.  Closed-world tracking of multiple interacting targets for indoor-sports applications , 2009, Comput. Vis. Image Underst..