Detection and localization of crowd behavior using a novel tracklet-based model

In this paper, two novel descriptors are introduced to detect and localize abnormal behaviors in crowded scenes. The first proposed descriptor is based on the orientation and magnitude of short trajectories extracted by tracking interest points in spatio-temporal 3D patches. The proposed descriptor employs a novel simplified Histogram of Oriented Tracklets (sHOT), which is shown to be very effective in the task of crowd abnormal behavior detection. In this scheme, abnormal behaviors are detected at different levels, namely spatio-temporal level and frame level. By combining the first proposed descriptor and the dense optical flow model, we propose our second framework which is able to localize the abnormal behavior areas in video sequences. The evaluation of our simple but yet effective descriptors on different state-of-the-art datasets, namely UCSD, UMN and Violence in Crowds yields very promising results in abnormality detection and outperforming different former state-of-the-art descriptors.

[1]  Dinesh Manocha,et al.  Realtime Multilevel Crowd Tracking Using Reciprocal Velocity Obstacles , 2014, 2014 22nd International Conference on Pattern Recognition.

[2]  Nicu Sebe,et al.  Learning Deep Representations of Appearance and Motion for Anomalous Event Detection , 2015, BMVC.

[3]  Mahmood Fathy,et al.  Video anomaly detection and localisation based on the sparsity and reconstruction error of auto-encoder , 2016 .

[4]  Christian Bauckhage,et al.  Loveparade 2010: Automatic video analysis of a crowd disaster , 2012, Comput. Vis. Image Underst..

[5]  Louis Kratz,et al.  Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models , 2009, CVPR.

[6]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[7]  Christian Bauckhage,et al.  Analyzing pedestrian behavior in crowds for automatic detection of congestions , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[8]  Wei Shen,et al.  Spatial-temporal convolutional neural networks for anomaly detection and localization in crowded scenes , 2016, Signal Process. Image Commun..

[9]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Alessandro Perina,et al.  Abnormality Detection with Improved Histogram of Oriented Tracklets , 2015, ICIAP.

[11]  Bo Wang,et al.  Abnormal crowd behavior detection using high-frequency and spatio-temporal features , 2011, Machine Vision and Applications.

[12]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[13]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[14]  Lior Wolf,et al.  Local Trinary Patterns for human action recognition , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[15]  Alessandro Perina,et al.  Analyzing Tracklets for the Detection of Abnormal Crowd Behavior , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[16]  Hamid R. Rabiee,et al.  Novel dataset for fine-grained abnormal behavior understanding in crowd , 2016, 2016 13th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[17]  Nuno Vasconcelos,et al.  Anomaly Detection and Localization in Crowded Scenes , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Mahmood Fathy,et al.  Real-time anomaly detection and localization in crowded scenes , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[19]  Alessandro Perina,et al.  Detecting Abnormal Behavioral Patterns in Crowd Scenarios , 2016, Toward Robotic Socially Believable Behaving Systems.

[20]  Nuno Vasconcelos,et al.  Anomaly detection in crowded scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Alessandro Perina,et al.  Crowd motion monitoring using tracklet-based commotion measure , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[22]  Ramakant Nevatia,et al.  Bayesian human segmentation in crowded situations , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[23]  Hau-San Wong,et al.  Crowd Motion Partitioning in a Scattered Motion Field , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[24]  Shaogang Gong,et al.  Cumulative Attribute Space for Age and Crowd Density Estimation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Bing Li,et al.  Efficient Clustering Aggregation Based on Data Fragments , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  Ko Nishino,et al.  Tracking with local spatio-temporal motion patterns in extremely crowded scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[27]  Stefano Soatto,et al.  Tracklet Descriptors for Action Modeling and Video Analysis , 2010, ECCV.

[28]  Wander Jager,et al.  Modelling Crowd dynamics, influence factors related to the probability of a riot , 2007 .

[29]  Cordelia Schmid,et al.  Action recognition by dense trajectories , 2011, CVPR 2011.

[30]  Nicu Sebe,et al.  Plug-and-Play CNN for Crowd Motion Analysis: An Application in Abnormal Event Detection , 2016, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[31]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Mubarak Shah,et al.  Detecting global motion patterns in complex videos , 2008, 2008 19th International Conference on Pattern Recognition.

[33]  Naixue Xiong,et al.  Abnormal event detection in crowded scenes based on deep learning , 2016, Multimedia Tools and Applications.

[34]  Mubarak Shah,et al.  Identifying Behaviors in Crowd Scenes Using Stability Analysis for Dynamical Systems , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Zhiwen Yu,et al.  A Bayesian Model for Crowd Escape Behavior Detection , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[36]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[37]  Junsong Yuan,et al.  Sparse reconstruction cost for abnormal event detection , 2011, CVPR 2011.

[38]  Xing Hu,et al.  Video anomaly detection using deep incremental slow feature analysis network , 2016, IET Comput. Vis..

[39]  Nicu Sebe,et al.  Emotion-Based Crowd Representation for Abnormality Detection , 2016, ArXiv.

[40]  Peter H. Tu,et al.  Simultaneous estimation of segmentation and shape , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[41]  Larry S. Davis,et al.  Action Recognition with Image Based CNN Features , 2015, ArXiv.

[42]  David G. Stork,et al.  Pattern Classification , 1973 .

[43]  Cordelia Schmid,et al.  Learning realistic human actions from movies , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Hossein Mousavi,et al.  Crowd behavior representation: an attribute-based approach , 2016, SpringerPlus.

[45]  Ramin Mehran,et al.  Abnormal crowd behavior detection using social force model , 2009, CVPR.

[46]  Tal Hassner,et al.  Violent flows: Real-time detection of violent crowd behavior , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[47]  Serge J. Belongie,et al.  Counting Crowded Moving Objects , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[48]  Hichem Snoussi,et al.  Histograms of Optical Flow Orientation for Visual Abnormal Events Detection , 2012, 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance.

[49]  Joachim Weickert,et al.  Reliable Estimation of Dense Optical Flow Fields with Large Displacements , 2000, International Journal of Computer Vision.

[50]  Ehud Rivlin,et al.  Robust Real-Time Unusual Event Detection using Multiple Fixed-Location Monitors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.