Abnormal Event Detection via Feature Expectation Subgraph Calibrating Classification in Video Surveillance Scenes

At present, the existing abnormal event detection models based on deep learning mainly focus on data represented by a vectorial form, which pay little attention to the impact of the internal structure characteristics of feature vector. In addition, a single classifier is difficult to ensure the accuracy of classification. In order to address the above issues, we propose an abnormal event detection hybrid modulation method via feature expectation subgraph calibrating classification in video surveillance scenes in this paper. Our main contribution is to calibrate the classification of a single classifier by constructing feature expectation subgraphs. First, we employ convolutional neural network and long short-term memory models to extract the spatiotemporal features of video frame, and then construct the feature expectation subgraph for each key frame of every video, which could be used to capture the internal sequential and topological relational characteristics of structured feature vector. Second, we project expectation subgraphs on the sparse vector to combine with a support vector classifier to calibrate the results of a linear support vector classifier. Finally, the experiments on a common dataset named UCSDped1 and a coal mining video dataset in comparison with some existing works demonstrate that the performance of the proposed method is better than several the state-of-the-art approaches.

[1]  Sridha Sridharan,et al.  Soft + Hardwired Attention: An LSTM Framework for Human Trajectory Prediction and Abnormal Event Detection , 2017, Neural Networks.

[2]  Deng Cai,et al.  Sparse Coding Guided Spatiotemporal Feature Learning for Abnormal Event Detection in Large Videos , 2019, IEEE Transactions on Multimedia.

[3]  Radu Tudor Ionescu,et al.  Deep Appearance Features for Abnormal Behavior Detection in Video , 2017, ICIAP.

[4]  Sudip Roy,et al.  CrowdVAS-Net: A Deep-CNN Based Framework to Detect Abnormal Crowd-Motion Behavior in Videos for Predicting Crowd Disaster , 2019, 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC).

[5]  Debi Prosad Dogra,et al.  Trajectory-Based Surveillance Analysis: A Survey , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  V. Vaidehi,et al.  Anomalous Event Detection in Traffic Video Based on Sequential Temporal Patterns of Spatial Interval Events , 2015, KSII Trans. Internet Inf. Syst..

[7]  Yong Liu,et al.  AnomalyNet: An Anomaly Detection Network for Video Surveillance , 2019, IEEE Transactions on Information Forensics and Security.

[8]  Yoshikazu Fukuyama,et al.  Unsupervised Fault Detection in Refrigeration Showcase with Single Class Data using Autoencoders , 2019 .

[9]  Wen Gao,et al.  Probabilistic Multi-Task Learning for Visual Saliency Estimation in Video , 2010, International Journal of Computer Vision.

[10]  Xuelong Li,et al.  Collective Representation for Abnormal Event Detection , 2017, Journal of Computer Science and Technology.

[11]  Yandong Tang,et al.  Video Anomaly Search in Crowded Scenes via Spatio-Temporal Motion Context , 2013, IEEE Transactions on Information Forensics and Security.

[12]  Lei Shu,et al.  Abnormal Event Detection Based on Saliency Information , 2015, MUE 2015.

[13]  Zhou Yang,et al.  Robust deep auto-encoding Gaussian process regression for unsupervised anomaly detection , 2020, Neurocomputing.

[14]  Wen-Nung Lie,et al.  Human Behavior Recognition from Multiview Videos , 2020, Inf. Sci..

[15]  Jeremy S. Smith,et al.  Abnormal Event Detection From Videos Using a Two-Stream Recurrent Variational Autoencoder , 2020, IEEE Transactions on Cognitive and Developmental Systems.

[16]  Wen-Nung Lie,et al.  Convolutional Recurrent Neural Networks for Posture Analysis in Fall Detection , 2018, J. Inf. Sci. Eng..

[17]  Rajib Rana,et al.  Phonocardiographic Sensing Using Deep Learning for Abnormal Heartbeat Detection , 2018, IEEE Sensors Journal.

[18]  Jinsul Kim,et al.  Applications of Anomaly Detection Using Deep Learning on Time Series Data , 2018, 2018 IEEE 16th Intl Conf on Dependable, Autonomic and Secure Computing, 16th Intl Conf on Pervasive Intelligence and Computing, 4th Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress(DASC/PiCom/DataCom/CyberSciTech).

[19]  Joo Kooi Tan,et al.  Temporal analysis for fast motion detection in a crowd , 2014, Artificial Life and Robotics.

[20]  Hichem Snoussi,et al.  Abnormal event detection based on analysis of movement information of video sequence , 2018 .

[21]  Yutong Zheng,et al.  Research on Data Enhanced Ancient Pictogram Recognition Method Based on Convolutional Neural Network , 2019 .

[22]  Nuno Vasconcelos,et al.  Anomaly detection in crowded scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Lin Wang,et al.  Abnormal Event Detection in Videos Using Hybrid Spatio-Temporal Autoencoder , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[24]  Li Sun,et al.  Event-based large scale surveillance video summarization , 2016, Neurocomputing.

[25]  Ting Li,et al.  Abnormal Event Detection Method in Surveillance Video Based on Temporal CNN and Sparse Optical Flow , 2019, ICCDE' 19.

[26]  Hichem Snoussi,et al.  Abnormal event detection via covariance matrix for optical flow based feature , 2018, Multimedia Tools and Applications.

[27]  Tao Xiang,et al.  Interestingness Prediction by Robust Learning to Rank , 2014, ECCV.

[28]  Ying Jiang,et al.  Discovering Graphical Visual Features for Abnormal Semantic Event Detection , 2017, Multimedia Tools and Applications.

[29]  Wen-Hsien Fang,et al.  Video anomaly detection and localization using hierarchical feature representation and Gaussian process regression , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Debi Prosad Dogra,et al.  Surveillance scene representation and trajectory abnormality detection using aggregation of multiple concepts , 2018, Expert Syst. Appl..

[31]  Vania Bogorny,et al.  Toward Abnormal Trajectory and Event Detection in Video Surveillance , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[32]  L. Kratz,et al.  Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Xuan Wang,et al.  Pedestrian abnormal event detection based on multi-feature fusion in traffic video , 2018 .

[34]  Fei-Fei Li,et al.  Online detection of unusual events in videos via dynamic sparse coding , 2011, CVPR 2011.

[35]  Jun Wang,et al.  Time Series Sequences Classification with Inception and LSTM Module , 2019, 2019 IEEE International Conference on Integrated Circuits, Technologies and Applications (ICTA).

[36]  Ashok Kumar P.M,et al.  Anomalous Event Detection in Traffic Video Based on Sequential Temporal Patterns of Spatial Interval Events , 2015 .

[37]  Damla Arifoglu,et al.  Detection of abnormal behaviour for dementia sufferers using Convolutional Neural Networks , 2019, Artif. Intell. Medicine.

[38]  Hichem Snoussi,et al.  Abnormal event detection using convolutional neural networks and 1-class SVM classifier , 2017, ICDP.

[39]  Rong Wang,et al.  Abnormal Event Detection by Learning Spatiotemporal Features in Videos , 2018, IGTA.

[40]  Yun Zhang,et al.  Near-Duplicate Video Cleansing Method Based on Locality Sensitive Hashing and the Sorted Neighborhood Method , 2019, 2nd EAI International Conference on Robotic Sensor Networks.

[41]  Xiaohui Hu,et al.  Finding Abnormal Vessel Trajectories Using Feature Learning , 2017, IEEE Access.

[42]  Naixue Xiong,et al.  Abnormal event detection in crowded scenes based on deep learning , 2016, Multimedia Tools and Applications.

[43]  Tiejun Huang,et al.  Sequential Deep Trajectory Descriptor for Action Recognition With Three-Stream CNN , 2016, IEEE Transactions on Multimedia.

[44]  Jie Shao,et al.  Abnormal event detection for video surveillance using deep one-class learning , 2017, Multimedia Tools and Applications.

[45]  Sridha Sridharan,et al.  Detecting rare events using Kullback-Leibler divergence: A weakly supervised approach , 2016, Expert Syst. Appl..

[46]  C. Krishna Mohan,et al.  Graph formulation of video activities for abnormal activity recognition , 2017, Pattern Recognit..

[47]  Antonio F. Gómez-Skarmeta,et al.  CEP-traj: An event-based solution to process trajectory data , 2015, Inf. Syst..

[48]  Chao Wang,et al.  Abnormal event detection based on cosparse reconstruction , 2018 .

[49]  Martin D. Levine,et al.  Online Dominant and Anomalous Behavior Detection in Videos , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Ezzeddine Zagrouba,et al.  Spatio-temporal feature using optical flow based distribution for violence detection , 2017, Pattern Recognit. Lett..

[51]  Cewu Lu,et al.  Abnormal Event Detection at 150 FPS in MATLAB , 2013, 2013 IEEE International Conference on Computer Vision.

[52]  Martial Hebert,et al.  A Discriminative Framework for Anomaly Detection in Large Videos , 2016, ECCV.

[53]  Ye Tao,et al.  A Novel Representation for Abnormal Crowd Motion Detection , 2017, IScIDE.

[54]  Amit Sethi,et al.  Abnormal Event Detection on BMTT-PETS 2017 Surveillance Challenge , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).