Deep-anomaly: Fully convolutional neural network for fast anomaly detection in crowded scenes

We present an efficient method for detecting and localizing anomalies in videos showing crowded scenes. Research on {\it fully convolutional neural networks} (FCNs) has shown the potentials of this technology for object detection and localization, especially in images. We investigate how to involve temporal data, and how to transform a supervised FCN into an unsupervised one such that the resulting FCN ensures anomaly detection. Altogether, we propose an FCN-based architecture for anomaly detection and localization in crowded scenes videos. For reducing computations and, consequently, improving performance both with respect to speed and accuracy, we investigate the use of cascaded out-layer detection. Our architecture includes two main components, one for feature representation, and one for cascaded out-layer detection. Experimental results on Subway and UCSD benchmarks confirm that the detection and localization accuracy of our method is comparable to state-of-the-art methods, but at a significantly increased speed of 370 fps.

[1]  Mubarak Shah,et al.  Abnormal crowd behavior detection using social force model , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Ivan Laptev,et al.  Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Junsong Yuan,et al.  Sparse reconstruction cost for abnormal event detection , 2011, CVPR 2011.

[4]  Xiaoqiang Lu,et al.  Learning deep event models for crowd anomaly detection , 2017, Neurocomputing.

[5]  Yandong Tang,et al.  Video Anomaly Search in Crowded Scenes via Spatio-Temporal Motion Context , 2013, IEEE Transactions on Information Forensics and Security.

[6]  Honglak Lee,et al.  An Analysis of Single-Layer Networks in Unsupervised Feature Learning , 2011, AISTATS.

[7]  Alberto Del Bimbo,et al.  Multi-scale and real-time non-parametric approach for anomaly detection and localization , 2012, Comput. Vis. Image Underst..

[8]  K. Grauman,et al.  Observe locally, infer globally: A space-time MRF for detecting abnormal activities with incremental updates , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Mubarak Shah,et al.  Chaotic invariants of Lagrangian particle trajectories for anomaly detection in crowded scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Martin D. Levine,et al.  Online Dominant and Anomalous Behavior Detection in Videos , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Jiwen Lu,et al.  PCANet: A Simple Deep Learning Baseline for Image Classification? , 2014, IEEE Transactions on Image Processing.

[14]  Aggelos K. Katsaggelos,et al.  Anomalous video event detection using spatiotemporal context , 2011 .

[15]  Nannan Li,et al.  Spatio-temporal context analysis within video volumes for anomalous-event detection and localization , 2015, Neurocomputing.

[16]  Mahmood Fathy,et al.  Real-time anomaly detection and localization in crowded scenes , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[17]  Björn Ommer,et al.  Video parsing for abnormality detection , 2011, 2011 International Conference on Computer Vision.

[18]  Andrei Zaharescu,et al.  Anomalous Behaviour Detection Using Spatiotemporal Oriented Energies, Subset Inclusion Histogram Comparison and Event-Driven Processing , 2010, ECCV.

[19]  Wen-Hsien Fang,et al.  Video anomaly detection and localization using hierarchical feature representation and Gaussian process regression , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Dong-Gyu Lee,et al.  Motion Influence Map for Unusual Human Activity Detection and Localization in Crowded Scenes , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[21]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[22]  Michal Irani,et al.  Detecting Irregularities in Images and in Video , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[23]  Brian C. Lovell,et al.  Improved anomaly detection in crowded scenes via cell-based analysis of foreground speed, size and texture , 2011, CVPR 2011 WORKSHOPS.

[24]  Mahmood Fathy,et al.  Deep-Cascade: Cascading 3D Deep Neural Networks for Fast Anomaly Detection and Localization in Crowded Scenes , 2017, IEEE Transactions on Image Processing.

[25]  Nuno Vasconcelos,et al.  Anomaly detection in crowded scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Gian Luca Foresti,et al.  Trajectory-Based Anomalous Event Detection , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Hongbin Zha,et al.  Learning to Detect Anomalies in Surveillance Video , 2015, IEEE Signal Processing Letters.

[28]  Gian Luca Foresti,et al.  On-line trajectory clustering for anomalous events detection , 2006, Pattern Recognit. Lett..

[29]  Cewu Lu,et al.  Abnormal Event Detection at 150 FPS in MATLAB , 2013, 2013 IEEE International Conference on Computer Vision.

[30]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[31]  Christophe Rosenberger,et al.  Abnormal events detection based on spatio-temporal co-occurences , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Samy Bengio,et al.  Semi-supervised adapted HMMs for unusual event detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[33]  Nuno Vasconcelos,et al.  Anomaly Detection and Localization in Crowded Scenes , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Nannan Li,et al.  Video anomaly detection based on a hierarchical activity discovery within spatio-temporal contexts , 2014, Neurocomputing.

[35]  Simone Calderara,et al.  Detecting anomalies in people's trajectories using spectral graph analysis , 2011, Comput. Vis. Image Underst..

[36]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[37]  Mahmood Fathy,et al.  Video anomaly detection and localisation based on the sparsity and reconstruction error of auto-encoder , 2016 .

[38]  Bolei Zhou,et al.  Learning Deep Features for Scene Recognition using Places Database , 2014, NIPS.

[39]  Nicola Conci,et al.  Dominant Motion Analysis in Regular and Irregular Crowd Scenes , 2014, HBU.

[40]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[41]  Martin D. Levine,et al.  An on-line, real-time learning method for detecting anomalies in videos using spatio-temporal compositions , 2013, Comput. Vis. Image Underst..

[42]  Ehud Rivlin,et al.  Robust Real-Time Unusual Event Detection using Multiple Fixed-Location Monitors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Nicu Sebe,et al.  Learning Deep Representations of Appearance and Motion for Anomalous Event Detection , 2015, BMVC.

[44]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[45]  Mohan M. Trivedi,et al.  Trajectory Learning for Activity Understanding: Unsupervised, Multilevel, and Long-Term Adaptive Approach , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Qi Wang,et al.  Online Anomaly Detection in Crowd Scenes via Structure Analysis , 2015, IEEE Transactions on Cybernetics.

[47]  Mahmood Fathy,et al.  Fast and accurate detection and localization of abnormal behavior in crowded scenes , 2017, Machine Vision and Applications.

[48]  Mubarak Shah,et al.  Semi-supervised Learning of Feature Hierarchies for Object Detection in a Video , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  David A. Clausi,et al.  Goal-based trajectory analysis for unusual behaviour detection in intelligent surveillance , 2011, Image Vis. Comput..

[50]  Alessandro Perina,et al.  Abnormality Detection with Improved Histogram of Oriented Tracklets , 2015, ICIAP.

[51]  Luca Maria Gambardella,et al.  Fast image scanning with deep max-pooling convolutional neural networks , 2013, 2013 IEEE International Conference on Image Processing.

[52]  Zhou Wang,et al.  On the Mathematical Properties of the Structural Similarity Index , 2012, IEEE Transactions on Image Processing.

[53]  Andrew Zisserman,et al.  Two-Stream Convolutional Networks for Action Recognition in Videos , 2014, NIPS.

[54]  Nicola Conci,et al.  Gaussian mixtures for anomaly detection in crowded scenes , 2013, Electronic Imaging.

[55]  Venkatesh Saligrama,et al.  Video anomaly detection based on local statistical aggregates , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[56]  Luc Van Gool,et al.  DeepProposal: Hunting Objects by Cascading Deep Convolutional Layers , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[57]  Naixue Xiong,et al.  Abnormal event detection in crowded scenes based on deep learning , 2016, Multimedia Tools and Applications.

[58]  Amit K. Roy-Chowdhury,et al.  Context-Aware Modeling and Recognition of Activities in Video , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[59]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[60]  Stavros J. Perantonis,et al.  Detecting abnormal human behaviour using multiple cameras , 2009, Signal Process..

[61]  L. Kratz,et al.  Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[62]  Tieniu Tan,et al.  A system for learning statistical motion patterns , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[63]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[64]  Nicola Conci,et al.  Real-time anomaly detection in dense crowded scenes , 2014, Electronic Imaging.