Unsupervised Adversarial Learning for Dynamic Background Modeling

Dynamic Background Modeling (DBM) is a crucial task in many computer vision based applications such as human activity analysis, traffic monitoring, surveillance, and security. DBM is extremely challenging in scenarios like illumination changes, camouflage, intermittent object motion or shadows. In this study, we proposed an end-to-end framework based on Generative Adversarial Network, which can generate dynamic background information for the task of DBM in an unsupervised manner. Our proposed model can handle the problem of DBM in the presence of the challenges mentioned above by generating data similar to the desired information. The primary aim of our proposed model during training is to learn all the dynamic changes in a scene-specific background information. While, during testing, inverse mapping of data to latent space representation in our model generates dynamic backgrounds similar to test data. The comparative analysis of our proposed model upon experimental evaluations on SBM.net and SBI benchmark datasets has outperformed eight existing methods for DBM in many challenging scenarios.

[1]  Soon Ki Jung,et al.  Moving Object Detection on RGB-D Videos Using Graph Regularized Spatiotemporal RPCA , 2017, ICIAP Workshops.

[2]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[3]  James M. Rehg,et al.  GOSUS: Grassmannian Online Subspace Updates with Structured-Sparsity , 2013, 2013 IEEE International Conference on Computer Vision.

[4]  Yong Liu,et al.  AnomalyNet: An Anomaly Detection Network for Video Surveillance , 2019, IEEE Transactions on Information Forensics and Security.

[5]  Laura Balzano,et al.  Incremental gradient on the Grassmannian for online foreground and background separation in subsampled video , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Hubert P. H. Shum,et al.  Northumbria Research , 2022 .

[7]  Sajid Javed,et al.  Robust Subspace Learning: Robust PCA, Robust Subspace Tracking, and Robust Subspace Recovery , 2017, IEEE Signal Processing Magazine.

[8]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[9]  Liqing Zhang,et al.  Bayesian Robust Tensor Factorization for Incomplete Multiway Data , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Soon Ki Jung,et al.  Background–Foreground Modeling Based on Spatiotemporal Sparse Subspace Clustering , 2017, IEEE Transactions on Image Processing.

[11]  Lavanya Sharma,et al.  Performance analysis of moving object detection using BGS techniques in visual surveillance , 2019, International Journal of Spatio-Temporal Data Science.

[12]  Yassine Ruichek,et al.  BSCGAN: Deep Background Subtraction with Conditional Generative Adversarial Networks , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[13]  Mario Ignacio Chacon Murguia,et al.  Temporal weighted learning model for background estimation with an automatic re-initialization stage and adaptive parameters update , 2017, Pattern Recognit. Lett..

[14]  Gerhard Rigoll,et al.  A deep convolutional neural network for video sequence background subtraction , 2018, Pattern Recognit..

[15]  Xiaochun Cao,et al.  Total Variation Regularized RPCA for Irregularly Moving Object Detection Under Dynamic Background , 2016, IEEE Transactions on Cybernetics.

[16]  Thierry Bouwmans,et al.  Background Subtraction in Real Applications: Challenges, Current Models and Future Directions , 2019, Comput. Sci. Rev..

[17]  Xiaowei Zhou,et al.  Moving Object Detection by Detecting Contiguous Outliers in the Low-Rank Representation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Chris Baber,et al.  Ecological Interface Design, the Proximity Compatibility Principle, and Automation Reliability in Road Traffic Management , 2019, IEEE Transactions on Human-Machine Systems.

[19]  Jianfei Cai,et al.  Background Subtraction Based on Deep Pixel Distribution Learning , 2018, 2018 IEEE International Conference on Multimedia and Expo (ICME).

[20]  Xiaochun Cao,et al.  Robust Foreground Detection Using Smoothness and Arbitrariness Constraints , 2014, ECCV.

[21]  Subrahmanyam Murala,et al.  FgGAN: A Cascaded Unpaired Learning for Background Estimation and Foreground Segmentation , 2019, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[22]  Z. Zivkovic Improved adaptive Gaussian mixture model for background subtraction , 2004, ICPR 2004.

[23]  Li Fei-Fei,et al.  Every Moment Counts: Dense Detailed Labeling of Actions in Complex Videos , 2015, International Journal of Computer Vision.

[24]  Soon Ki Jung,et al.  Spatiotemporal Low-Rank Modeling for Complex Scene Background Initialization , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[25]  Minh N. Do,et al.  Semantic Image Inpainting with Deep Generative Models , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[27]  Ferdinand van der Heijden,et al.  Efficient adaptive density estimation per image pixel for the task of background subtraction , 2006, Pattern Recognit. Lett..

[28]  Soon Ki Jung,et al.  Unsupervised deep context prediction for background estimation and foreground segmentation , 2018, Machine Vision and Applications.

[29]  Lucia Maddalena,et al.  Towards Benchmarking Scene Background Initialization , 2015, ICIAP Workshops.

[30]  Thierry Bouwmans,et al.  Robust PCA via Principal Component Pursuit: A review for a comparative evaluation in video surveillance , 2014, Comput. Vis. Image Underst..

[31]  Marc Van Droogenbroeck,et al.  LaBGen-P-Semantic: A First Step for Leveraging Semantic Segmentation in Background Generation , 2018, J. Imaging.

[32]  Ezzeddine Zagrouba,et al.  Abnormal behavior recognition for intelligent video surveillance systems: A review , 2018, Expert Syst. Appl..

[33]  Andrew M. Dai,et al.  Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step , 2017, ICLR.

[34]  Lucia Maddalena,et al.  Scene background initialization: A taxonomy , 2017, Pattern Recognit. Lett..

[35]  Soon Ki Jung,et al.  Deep Neural Network Concepts for Background Subtraction: A Systematic Review and Comparative Evaluation , 2018, Neural Networks.

[36]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[37]  Mubarak Shah,et al.  Real-World Anomaly Detection in Surveillance Videos , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.