Deep and Sparse features For Anomaly Detection and Localization in video

Detection of abnormality in video crowded scenes is highly challenging in computer vision over the past decades. We propose a method for the detection of abnormal events in surveillance video sequences. In this method, we combine the spatial-temporal convolution neural network (CNN) with handcrafted feature sets such as Histograms of Optical Flow (HOF) and Histogram of Oriented Gradients (HOG) for anomaly detection in contiguous video frames. Handcrafted features learned sparse by using our novel method which we named it Iterative Weighted nonNegative Matrix Factorization (IW-NMF) is based on sparse NMF. These feature extracted from active volume cells that included moving pixels to reduce computational costs. The architecture of the CNN model allows us to extract spatial-temporal features and using handcrafted features to increase the accuracy of detection and to ensure robustness against local noise. we evaluate our framework on popular datasets that contain abnormal activites. our method results are better than most of the other method. and achieves a very competitive detection performance compared to state-of-the-art methods.

[1]  Graham Coleman,et al.  Detection and explanation of anomalous activities: representing activities as bags of event n-grams , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[2]  Venkatesh Saligrama,et al.  Video anomaly detection based on local statistical aggregates , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Hong-Yuan Mark Liao,et al.  Learning deep and sparse feature representation for fine-grained object recognition , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[4]  Kristen Grauman,et al.  Observe locally, infer globally: A space-time MRF for detecting abnormal activities with incremental updates , 2009, CVPR.

[5]  Jianhua Liu,et al.  Deep Sparse Autoencoder for Feature Extraction and Diagnosis of Locomotive Adhesion Status , 2018, J. Control. Sci. Eng..

[6]  Wei Shen,et al.  Spatial-temporal convolutional neural networks for anomaly detection and localization in crowded scenes , 2016, Signal Process. Image Commun..

[7]  Mubarak Shah,et al.  Chaotic invariants of Lagrangian particle trajectories for anomaly detection in crowded scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Ivan Laptev,et al.  On Space-Time Interest Points , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[9]  Wen-Hsien Fang,et al.  Gaussian Process Regression-Based Video Anomaly Detection and Localization With Hierarchical Feature Representation , 2015, IEEE Transactions on Image Processing.

[10]  Jianbo Shi,et al.  Detecting unusual activity in video , 2004, CVPR 2004.

[11]  Michal Irani,et al.  Detecting Irregularities in Images and in Video , 2005, ICCV.

[12]  Yandong Tang,et al.  Video Anomaly Search in Crowded Scenes via Spatio-Temporal Motion Context , 2013, IEEE Transactions on Information Forensics and Security.

[13]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[14]  Martin D. Levine,et al.  An on-line, real-time learning method for detecting anomalies in videos using spatio-temporal compositions , 2013, Comput. Vis. Image Underst..

[15]  D. Sculley,et al.  Web-scale k-means clustering , 2010, WWW '10.

[16]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[18]  Ramin Mehran,et al.  Abnormal crowd behavior detection using social force model , 2009, CVPR.

[19]  Larry S. Davis,et al.  Unsupervised Abnormal Crowd Activity Detection Using Semiparametric Scan Statistic , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[20]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Nanning Zheng,et al.  Non-negative matrix factorization for visual coding , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[22]  Yan Wang,et al.  DeepContour: A deep convolutional feature learned by positive-sharing loss for contour detection , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Sebastian Scherer,et al.  3D Convolutional Neural Networks for landing zone detection from LiDAR , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[24]  V. Abolghasemi,et al.  Iterative Weighted Non-smooth Non-negative Matrix Factorization for Face Recognition , 2018, International Journal of Engineering.

[25]  Michael G. Strintzis,et al.  Swarm Intelligence for Detecting Interesting Events in Crowded Environments , 2015, IEEE Transactions on Image Processing.

[26]  Nuno Vasconcelos,et al.  Anomaly Detection and Localization in Crowded Scenes , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Shaogang Gong,et al.  Global Behaviour Inference using Probabilistic Latent Semantic Analysis , 2008, BMVC.

[28]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[29]  Junsong Yuan,et al.  Abnormal event detection in crowded scenes using sparse representation , 2013, Pattern Recognit..

[30]  Junsong Yuan,et al.  Sparse reconstruction cost for abnormal event detection , 2011, CVPR 2011.

[31]  Aggelos K. Katsaggelos,et al.  Anomalous video event detection using spatiotemporal context , 2011 .

[32]  Ehud Rivlin,et al.  Robust Real-Time Unusual Event Detection using Multiple Fixed-Location Monitors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[34]  W. Eric L. Grimson,et al.  Unsupervised Activity Perception by Hierarchical Bayesian Models , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Meng Wang,et al.  3D Human Activity Recognition with Reconfigurable Convolutional Neural Networks , 2014, ACM Multimedia.

[36]  Mubarak Shah,et al.  Learning object motion patterns for anomaly detection and improved object detection , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Nannan Li,et al.  Video anomaly detection based on a hierarchical activity discovery within spatio-temporal contexts , 2014, Neurocomputing.

[38]  Chang-Tsun Li,et al.  Video Anomaly Detection With Compact Feature Sets for Online Performance , 2017, IEEE Transactions on Image Processing.

[39]  Changsheng Li,et al.  Sparse representation for robust abnormality detection in crowded scenes , 2014, Pattern Recognit..

[40]  Mahmood Fathy,et al.  Real-time anomaly detection and localization in crowded scenes , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[41]  Cewu Lu,et al.  Abnormal Event Detection at 150 FPS in MATLAB , 2013, 2013 IEEE International Conference on Computer Vision.