NM-GAN: Noise-modulated generative adversarial network for video anomaly detection

Abstract As an important and challenging task for intelligent video surveillance systems, video anomaly detection is generally referred to as automatic recognition of video frames that contain abnormal targets, behavior or events. Although it has been widely applied in real scenes, anomaly detection remains a challenging task because of the vague definition of anomaly and the lack of the anomaly samples. Inspired by the widespread application of Generative Adversarial Network (GAN), we propose an end-to-end pipeline called NM-GAN which assembles an encode-decoder reconstruction network and a CNN-based discrimination network in a GAN-like architecture. The generalization ability of the reconstruction network is properly modulated via the adversarial learning around reconstruction error maps and noise maps. Meanwhile, the discrimination network is trained to distinguish anomaly samples from normal samples based on the reconstruction error maps. Finally, the output of the discrimination network is transferred to evaluate anomaly score of the input frame. The thorough proof-of-principle experiments and ablation tests on several popular datasets reveal that the proposed model enhance the generalization ability of the reconstruction network and the distinguishability of the discrimination network significantly. The comparison with the state-of-the-art shows that the proposed NM-GAN model outperforms most competing models in precision and stability.