Video anomaly detection with multi-scale feature and temporal information fusion

Abstract Video anomaly detection is a challenging task because of the uncertainty of abnormal events. The current method based on predictive frames has obtained better detection results compared with the previous reconstruction or hand-crafted methods. In current prediction methods, the characteristics considered previously are only of a single scale, and the time constraint information is not fully used. In our work, we proposed a new framework structure to achieve better abnormality detection rate. To address the objects of different scales in each video frame, we considered extracting the characteristics of different receptive fields to encode more spatial information. At the same time, we added temporal constraints to the network instead of using time-consuming optical flow information, and we completed the memory of temporal features through a ConvGRU module. Furthermore, while distinguishing abnormal events, we also considered temporal information and spatial information so that our framework could fully combine spatio-temporal information to correctly distinguish abnormal events from normal events. We obtained excellent results on three datasets, thus demonstrating the effectiveness of our method.