Learning deep event models for crowd anomaly detection

Abnormal event detection in video surveillance is extremely important, especially for crowded scenes. In recent years, many algorithms have been proposed based on hand-crafted features. However, it still remains challenging to decide which kind of feature is suitable for a specific situation. In addition, it is hard and time-consuming to design an effective descriptor. In this paper, video events are automatically represented and modeled in unsupervised fashions. Specifically, appearance and motion features are simultaneously extracted using a PCANet from 3D gradients. In order to model event patterns, a deep Gaussian mixture model (GMM) is constructed with observed normal events. The deep GMM is a scalable deep generative model which stacks multiple GMM-layers on top of each other. As a result, the proposed method acquires competitive performance with relatively few parameters. In the testing phase, the likelihood is calculated to judge whether a video event is abnormal or not. In this paper, the proposed method is verified on two publicly available datasets and compared with state-of-the-art algorithms. Experimental results show that the deep model is effective for abnormal event detection in video surveillance.

[1]  Louis Kratz,et al.  Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models , 2009, CVPR.

[2]  Kristen Grauman,et al.  Observe locally, infer globally: A space-time MRF for detecting abnormal activities with incremental updates , 2009, CVPR.

[3]  Lorenzo Torresani,et al.  Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[4]  Nannan Li,et al.  Spatio-temporal context analysis within video volumes for anomalous-event detection and localization , 2015, Neurocomputing.

[5]  Qixiang Ye,et al.  Visual abnormal behavior detection based on trajectory sparse reconstruction analysis , 2013, Neurocomputing.

[6]  Cewu Lu,et al.  Abnormal Event Detection at 150 FPS in MATLAB , 2013, 2013 IEEE International Conference on Computer Vision.

[7]  Fei-Fei Li,et al.  Online detection of unusual events in videos via dynamic sparse coding , 2011, CVPR 2011.

[8]  Mubarak Shah,et al.  Learning object motion patterns for anomaly detection and improved object detection , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Brian C. Lovell,et al.  Improved anomaly detection in crowded scenes via cell-based analysis of foreground speed, size and texture , 2011, CVPR 2011 WORKSHOPS.

[10]  Ramin Mehran,et al.  Abnormal crowd behavior detection using social force model , 2009, CVPR.

[11]  Nannan Li,et al.  Video anomaly detection based on a hierarchical activity discovery within spatio-temporal contexts , 2014, Neurocomputing.

[12]  Andrea Cavallaro,et al.  Multifeature Object Trajectory Clustering for Video Analysis , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Naixue Xiong,et al.  Abnormal event detection in crowded scenes based on deep learning , 2016, Multimedia Tools and Applications.

[14]  Junsong Yuan,et al.  Abnormal event detection in crowded scenes using sparse representation , 2013, Pattern Recognit..

[15]  Palaiahnakote Shivakumara,et al.  Anomaly Detection through Spatio-temporal Context Modeling in Crowded Scenes , 2014, 2014 22nd International Conference on Pattern Recognition.

[16]  Ehud Rivlin,et al.  Robust Real-Time Unusual Event Detection using Multiple Fixed-Location Monitors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Zhongfei Zhang,et al.  Deep learning driven blockwise moving object detection with binary scene modeling , 2015, Neurocomputing.

[18]  Jenq-Neng Hwang,et al.  Integrated video object tracking with applications in trajectory-based event detection , 2011, J. Vis. Commun. Image Represent..

[19]  Nannan Li,et al.  Quaternion discrete cosine transformation signature analysis in crowd scenes for abnormal event detection , 2016, Neurocomputing.

[20]  Kejun Wang,et al.  Video-Based Abnormal Human Behavior Recognition—A Review , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[21]  Hong Liu,et al.  Depth Context: a new descriptor for human activity recognition by using sole depth sequences , 2016, Neurocomputing.

[22]  Paolo Remagnino,et al.  Laplacian Eigenmap With Temporal Constraints for Local Abnormality Detection in Crowded Scenes , 2013, IEEE Transactions on Cybernetics.

[23]  Junsong Yuan,et al.  Sparse reconstruction cost for abnormal event detection , 2011, CVPR 2011.

[24]  Qingshan Liu,et al.  Abnormal detection using interaction energy potentials , 2011, CVPR 2011.

[25]  Nicu Sebe,et al.  Learning Deep Representations of Appearance and Motion for Anomalous Event Detection , 2015, BMVC.

[26]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Benjamin Schrauwen,et al.  Factoring Variations in Natural Images with Deep Gaussian Mixture Models , 2014, NIPS.

[28]  Qi Wang,et al.  Part-Based Online Tracking With Geometry Constraint and Attention Selection , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[29]  Mubarak Shah,et al.  Chaotic invariants of Lagrangian particle trajectories for anomaly detection in crowded scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Dit-Yan Yeung,et al.  Learning a Deep Compact Image Representation for Visual Tracking , 2013, NIPS.

[31]  Qingming Huang,et al.  Abnormal event detection in crowded scenes based on Structural Multi-scale Motion Interrelated Patterns , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[32]  Jiwen Lu,et al.  PCANet: A Simple Deep Learning Baseline for Image Classification? , 2014, IEEE Transactions on Image Processing.

[33]  Brett J. Borghetti,et al.  A Review of Anomaly Detection in Automated Surveillance , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[34]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[35]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[36]  Michael S. Lew,et al.  Deep learning for visual understanding: A review , 2016, Neurocomputing.

[37]  Nuno Vasconcelos,et al.  Anomaly detection in crowded scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[38]  Gian Luca Foresti,et al.  Trajectory-Based Anomalous Event Detection , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[39]  Nuno Vasconcelos,et al.  Anomaly Detection and Localization in Crowded Scenes , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.