Abnormal event detection in crowded scenes based on deep learning

In this paper, we propose to use the deep learning technique for abnormal event detection by extracting spatiotemporal features from video sequences. Human eyes are often attracted to abnormal events in video sequences, thus we firstly extract saliency information (SI) of video frames as the feature representation in the spatial domain. Optical flow (OF) is estimated as an important feature of video sequences in the temporal domain. To extract the accurate motion information, multi-scale histogram optical flow (MHOF) can be obtained through OF. We combine MHOF and SI into the spatiotemporal features of video frames. Finally a deep learning network, PCANet, is adopted to extract high-level features for abnormal event detection. Experimental results show that the proposed abnormal event detection method can obtain much better performance than the existing ones on the public video database.

[1]  Nicu Sebe,et al.  Image saliency by isocentric curvedness and color , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[2]  Mohammad Mehdi Homayounpour,et al.  Normal sparse Deep Belief Network , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[3]  Christophe Rosenberger,et al.  Abnormal events detection based on spatio-temporal co-occurences , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Bastian Leibe,et al.  Tracking People and Their Objects , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Junseok Kwon,et al.  Minimum Uncertainty Gap for Robust Visual Tracking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Jiwen Lu,et al.  PCANet: A Simple Deep Learning Baseline for Image Classification? , 2014, IEEE Transactions on Image Processing.

[7]  Ming-Hsuan Yang,et al.  Top-down visual saliency via joint CRF and dictionary learning , 2012, CVPR.

[8]  Ramin Mehran,et al.  Abnormal crowd behavior detection using social force model , 2009, CVPR.

[9]  Tal Hassner,et al.  Violent flows: Real-time detection of violent crowd behavior , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[10]  Xiaogang Wang,et al.  Deep Convolutional Network Cascade for Facial Point Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Paolo Remagnino,et al.  Laplacian Eigenmap With Temporal Constraints for Local Abnormality Detection in Crowded Scenes , 2013, IEEE Transactions on Cybernetics.

[12]  Yandong Tang,et al.  Video Anomaly Search in Crowded Scenes via Spatio-Temporal Motion Context , 2013, IEEE Transactions on Information Forensics and Security.

[13]  Robert B. Fisher,et al.  Modelling Crowd Scenes for Event Detection , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[14]  Real-Time Detection of Violent Crowd Behavior , .

[15]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[16]  Ying Wu,et al.  Discriminative Video Pattern Search for Efficient Action Detection , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[18]  Shoab A. Khan,et al.  Tracking and Abnormal Behavior Detection in Video Surveillance Using Optical Flow and Neural Networks , 2014, 2014 28th International Conference on Advanced Information Networking and Applications Workshops.

[19]  Jean-Marc Odobez,et al.  Topic models for scene analysis and abnormality detection , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[20]  Bu-Sung Lee,et al.  Bottom-Up Saliency Detection Model Based on Human Visual Sensitivity and Amplitude Spectrum , 2012, IEEE Transactions on Multimedia.

[21]  Thomas Deselaers,et al.  What is an object? , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Hichem Snoussi,et al.  Detection of Abnormal Visual Events via Global Optical Flow Orientation Histogram , 2014, IEEE Transactions on Information Forensics and Security.

[23]  Liming Zhang,et al.  A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression , 2010, IEEE Transactions on Image Processing.

[24]  Yibo Li,et al.  Abnormal Event Detection in Nature Settings , 2014 .

[25]  Weisi Lin,et al.  Perceptual Full-Reference Quality Assessment of Stereoscopic Images by Considering Binocular Visual Characteristics , 2013, IEEE Transactions on Image Processing.

[26]  Qingming Huang,et al.  Abnormal crowd behavior detection based on social attribute-aware force model , 2012, 2012 19th IEEE International Conference on Image Processing.

[27]  Vibhav Vineet,et al.  Efficient Salient Region Detection with Soft Image Abstraction , 2013, 2013 IEEE International Conference on Computer Vision.

[28]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Afshin Dehghan,et al.  Improving an Object Detector and Extracting Regions Using Superpixels , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Aini Hussain,et al.  Sudden Event Recognition: A Survey , 2013, Sensors.

[31]  Wan-Young Chung,et al.  Visual Sensor Based Abnormal Event Detection with Moving Shadow Removal in Home Healthcare Applications , 2012, Sensors.

[32]  Qingshan Liu,et al.  Abnormal detection using interaction energy potentials , 2011, CVPR 2011.

[33]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[34]  R. Suganya,et al.  Data Mining Concepts and Techniques , 2010 .

[35]  Jie Chen,et al.  Online Least Squares One-Class Support Vector Machines-Based Abnormal Visual Event Detection , 2013, Sensors.

[36]  Hang-Bong Kang,et al.  Integrated multiple behavior models for abnormal crowd behavior detection , 2012, 2012 IEEE Southwest Symposium on Image Analysis and Interpretation.

[37]  Christophe Rosenberger,et al.  Abnormal events detection based on spatio-temporal co-occurences , 2009, CVPR.

[38]  Mubarak Shah,et al.  A Streakline Representation of Flow in Crowded Scenes , 2010, ECCV.

[39]  Deva Ramanan,et al.  Histograms of Sparse Codes for Object Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Xiaogang Wang,et al.  Deep Learning Face Representation from Predicting 10,000 Classes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[42]  Kejun Wang,et al.  Video-Based Abnormal Human Behavior Recognition—A Review , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[43]  Ehud Rivlin,et al.  Robust Real-Time Unusual Event Detection using Multiple Fixed-Location Monitors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[45]  Deepu Rajan,et al.  Salient Region Detection by Modeling Distributions of Color and Orientation , 2009, IEEE Transactions on Multimedia.

[46]  Deva Ramanan,et al.  Self-Paced Learning for Long-Term Tracking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Rongrong Ji,et al.  Social Attribute-Aware Force Model: Exploiting Richness of Interaction for Abnormal Crowd Detection , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[49]  Weisi Lin,et al.  Blind Image Quality Assessment for Stereoscopic Images Using Binocular Guided Quality Lookup and Visual Codebook , 2015, IEEE Transactions on Broadcasting.

[50]  Ming Dong,et al.  Real-time detection of abnormal crowd behavior using a matrix approximation-based approach , 2012, 2012 19th IEEE International Conference on Image Processing.

[51]  Mohammad Pezeshki,et al.  Deep Belief Networks for Image Denoising , 2014, ICLR.

[52]  L. Li,et al.  On pixel count based crowd density estimation for visual surveillance , 2004, IEEE Conference on Cybernetics and Intelligent Systems, 2004..

[53]  Mei Yu,et al.  Binocular energy response based quality assessment of stereoscopic images , 2014, Digit. Signal Process..

[54]  Lu Zhang,et al.  Structure Preserving Object Tracking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Liming Zhang,et al.  Spatio-temporal Saliency detection using phase spectrum of quaternion fourier transform , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.