Event Based Switched Dynamic Bayesian Networks for Autonomous Cognitive Crowd Monitoring

Human behavior analysis is one of the most important applications in Intelligent Video Surveillance (IVS) field. In most recent systems addressed by research, automatic support to the human decisions based on object detection, tracking and situation assessment tools is integrated as a part of a complete cognitive artificial process including security maintenance procedures actions that are in the scope of the system. In such cases an IVS needs to represent complex situations that describe alternative possible real time interactions between the dynamic observed situation and operators’ actions. To obtain such knowledge, particular types of Event based Dynamic Bayesian Networks E-DBNs are here proposed that can switch among alternative Bayesian filtering and control lower level modules to capture adaptive reactions of human operators. It is shown that after the off line learning phase Switched E-DBNs can be used to represent and anticipate possible operators’ actions within the IVS. In this sense acquired knowledge can be used for either fully autonomous security preserving systems or for training of new operators. Results are shown by considering a crowd monitoring application in a critical infrastructure. A system is presented where a Cognitive Node (CN) embedding in a structured way Switched E-DBN knowledge can interact with an active visual simulator of crowd situations. It is also shown that outputs from such a simulator can be easily compared with video signals coming from real cameras and processed by typical Bayesian tracking methods.

[1]  Sergio A. Velastin,et al.  Intelligent distributed surveillance systems: a review , 2005 .

[2]  Kai Oliver Arras,et al.  People tracking with human motion predictions from social forces , 2010, 2010 IEEE International Conference on Robotics and Automation.

[3]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Luc Van Gool,et al.  You'll never walk alone: Modeling social behavior for multi-target tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[5]  Andrea F. Cattoni,et al.  Interaction Modeling and Prediction in Smart Spaces: A Bio-Inspired Approach Based on Autobiographical Memory , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[6]  A. Damasio.  The Feeling of What Happens: Body and Emotion in the Making of Consciousness , 1999 .

[7]  Jayan Eledath,et al.  Collision sensing by stereo vision and radar sensor fusion , 2008 .

[8]  Carlo S. Regazzoni,et al.  Bayesian Bio-inspired Model for Learning Interactive Trajectories , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[9]  Pramod K. Varshney,et al.  Multisensor Surveillance Systems , 2003, Springer US.

[10]  Ramin Mehran,et al.  Abnormal crowd behavior detection using social force model , 2009, CVPR.

[11]  Céline Loscos,et al.  Intuitive crowd behavior in dense urban environments using local laws , 2003, Proceedings of Theory and Practice of Computer Graphics, 2003..

[12]  M. Nixon,et al.  On crowd density estimation for surveillance , 2006 .

[13]  Takeo Kanade,et al.  Algorithms for cooperative multisensor surveillance , 2001, Proc. IEEE.

[14]  Chabane Djeraba,et al.  Motion Pattern Extraction and Event Detection for Automatic Visual Surveillance , 2011, EURASIP J. Image Video Process..

[15]  Mubarak Shah,et al.  Visual crowd surveillance through a hydrodynamics lens , 2011, Commun. ACM.

[16]  Zhen Liu,et al.  A Simulation Based on Emotions Model for Virtual Human Crowds , 2009, 2009 Fifth International Conference on Image and Graphics.

[17]  Chung-Ping Young,et al.  Intelligent data fusion system for predicting vehicle collision warning using vision/GPS sensing , 2010, Expert Syst. Appl..

[18]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Mohan M. Trivedi,et al.  Intelligent environments and active camera networks , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[20]  Mohan M. Trivedi,et al.  Novel concepts and challenges for the next generation of video surveillance systems , 2007, Machine Vision and Applications.

[21]  Sergio A. Velastin,et al.  Crowd monitoring using image processing , 1995 .

[22]  M. Thonnat,et al.  Video understanding for metro surveillance , 2004, IEEE International Conference on Networking, Sensing and Control, 2004.

[23]  Luca Benini,et al.  An integrated multi-modal sensor network for video surveillance , 2005, VSSN '05.

[24]  Ramakant Nevatia,et al.  Bayesian human segmentation in crowded situations , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[25]  P. Varshney,et al.  Multisensor surveillance systems : the fusion perspective , 2003 .

[26]  Luciano da Fontoura Costa,et al.  Automatic estimation of crowd density using texture , 1998 .

[27]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[28]  Robert B. Fisher,et al.  Hidden Markov Models for Optical Flow Analysis in Crowds , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[29]  Alex Rogers,et al.  Modelling Driver Interdependent Behaviour in Agent-Based Traffic Simulations for Disaster Management , 2011, PAAMS.

[30]  Alex Pentland,et al.  Modeling Dynamical Influence in Human Interaction: Using data to make better inferences about influence within social systems , 2012, IEEE Signal Processing Magazine.

[31]  Chansoo Kim,et al.  Rician nonlocal means denoising for MR images using nonparametric principal component analysis , 2011, EURASIP J. Image Video Process..

[32]  A. Damasio.  The feeling of what happens , 2001 .

[33]  Tarak Gandhi,et al.  Looking-In and Looking-Out of a Vehicle: Computer-Vision-Based Enhanced Vehicle Safety , 2007, IEEE Transactions on Intelligent Transportation Systems.

[34]  Naren Ramakrishnan,et al.  Discovering Excitatory Networks from Discrete Event Streams with Applications to Neuronal Spike Train Analysis , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[35]  Carlo S. Regazzoni,et al.  Bayesian Tracking for Video Analytics , 2010, IEEE Signal Processing Magazine.

[36]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[37]  Sameer Singh,et al.  Approaches to Multisensor Data Fusion in Target Tracking: A Survey , 2006, IEEE Transactions on Knowledge and Data Engineering.

[38]  Alan J. Lipton,et al.  Automated video protection, monitoring & detection , 2003 .

[39]  Sergio A. Velastin,et al.  Crowd analysis: a survey , 2008, Machine Vision and Applications.