A novel background subtraction algorithm based on parallel vision and Bayesian GANs

Abstract To address the challenges of change detection in the wild, we present a novel background subtraction algorithm based on parallel vision and Bayesian generative adversarial networks (GANs). First, we use the median filtering algorithm for background image extraction. Then, we build the background subtraction model by using Bayesian GANs to classify all pixels into foreground and background, and use parallel vision theory to improve the background subtraction results in complex scenes. The proposed algorithm has been evaluated on the well-known, publicly available changedetection.net dataset. Experiment results show that the proposed algorithm results in better performance than many state-of-the-art ones. In addition, our model trained on CDnet dataset can generalize very well to unseen datasets, outperforming multiple state-of-art methods. The major contribution of this work is to apply parallel vision and Bayesian GANs to solve the difficulties in background subtraction, achieving high detection accuracy.

[1]  Gerhard Rigoll,et al.  A deep convolutional neural network for video sequence background subtraction , 2018, Pattern Recognit..

[2]  Jian Sun,et al.  Convolutional neural networks at constrained time cost , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Li Li,et al.  Steps toward Parallel Intelligence , 2016 .

[4]  Shiqi Yu,et al.  GaitGAN: Invariant Gait Feature Extraction Using Generative Adversarial Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[5]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[6]  Bo Li,et al.  Vehicle license plate recognition based on class-specific ERs and SaE-ELM , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[7]  Fei-Yue Wang,et al.  Toward a Paradigm Shift in Social Computing: The ACP Approach , 2007, IEEE Intell. Syst..

[8]  Jamie P. Heather Calibration of an industrial vision system using an ellipsoid , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[9]  Francesco Tisato,et al.  Attentive Monitoring of Multiple Video Streams Driven by a Bayesian Foraging Strategy , 2014, IEEE Transactions on Image Processing.

[10]  G. Sreelekha,et al.  Background subtraction with outlier replacement , 2015, 2015 IEEE Recent Advances in Intelligent Computational Systems (RAICS).

[11]  Xuan Li,et al.  Training and testing object detectors with virtual images , 2017, IEEE/CAA Journal of Automatica Sinica.

[12]  Fei-Yue Wang,et al.  $M^{4}CD$ : A Robust Change Detection Method for Intelligent Visual Surveillance , 2018, IEEE Access.

[13]  Fei-Yue Wang,et al.  Generative adversarial networks: introduction and outlook , 2017, IEEE/CAA Journal of Automatica Sinica.

[14]  Lin Chen,et al.  An end-to-end generative adversarial network for crowd counting under complicated scenes , 2017, 2017 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB).

[15]  Yong Yang,et al.  Variational Bayesian learning for background subtraction based on local fusion feature , 2016, IET Comput. Vis..

[16]  Fei-Yue Wang,et al.  A Multi-view Learning Approach to Foreground Detection for Traffic Surveillance Applications , 2016, IEEE Transactions on Vehicular Technology.

[17]  Haibo He,et al.  Online Learning Control Using Adaptive Critic Designs With Sparse Kernel Machines , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[18]  David Windridge,et al.  Multilevel Chinese Takeaway Process and Label-Based Processes for Rule Induction in the Context of Automated Sports Video Annotation , 2014, IEEE Transactions on Cybernetics.

[19]  Long Ang Lim,et al.  Foreground segmentation using convolutional neural networks for multiscale feature encoding , 2018, Pattern Recognit. Lett..

[20]  Nuno Vasconcelos,et al.  Anomaly Detection and Localization in Crowded Scenes , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Xiaowei Zhou,et al.  Moving Object Detection by Detecting Contiguous Outliers in the Low-Rank Representation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Sahin Isik,et al.  SWCD: a sliding window and self-regulated learning-based background updating method for change detection in videos , 2018, J. Electronic Imaging.

[23]  Bo Tang,et al.  Semisupervised Feature Selection Based on Relevance and Redundancy Criteria , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[24]  Yoshua Bengio,et al.  Drawing and Recognizing Chinese Characters with Recurrent Neural Network , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Marc Van Droogenbroeck,et al.  Semantic background subtraction , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[26]  Jean Meunier,et al.  Comparative study of motion detection methods for video surveillance systems , 2017, J. Electronic Imaging.

[27]  Lucia Maddalena,et al.  Towards Benchmarking Scene Background Initialization , 2015, ICIAP Workshops.

[28]  Bhaskara Marthi,et al.  A generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs , 2017, Science.

[29]  Jean-Yves Tourneret,et al.  A Hamiltonian Monte Carlo Method for Non-Smooth Energy Sampling , 2014, IEEE Transactions on Signal Processing.

[30]  Lucia Maddalena,et al.  Scene background initialization: A taxonomy , 2017, Pattern Recognit. Lett..

[31]  Nuno Vasconcelos,et al.  Anomaly detection in crowded scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[32]  Hasan Sajid,et al.  Universal Multimode Background Subtraction , 2017, IEEE Transactions on Image Processing.

[33]  Zhiming Luo,et al.  Interactive deep learning method for segmenting moving objects , 2017, Pattern Recognit. Lett..

[34]  Narciso García,et al.  Real-time nonparametric background subtraction with tracking-based foreground update , 2018, Pattern Recognit..

[35]  Nanning Zheng,et al.  Parallel vision for perception and understanding of complex scenes: methods, framework, and perspectives , 2017, Artificial Intelligence Review.

[36]  Guillaume-Alexandre Bilodeau,et al.  Improving background subtraction using Local Binary Similarity Patterns , 2014, IEEE Winter Conference on Applications of Computer Vision.

[37]  Rui Wang,et al.  Static and Moving Object Detection Using Flux Tensor with Split Gaussian Models , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[38]  Wen Gao,et al.  Background Subtraction via generalized fused lasso foreground modeling , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Dayong Shen,et al.  Visual Tracking Based on Dynamic Coupled Conditional Random Field Model , 2016, IEEE Transactions on Intelligent Transportation Systems.

[40]  Hanqing Lu,et al.  Learning sharable models for robust background subtraction , 2015, 2015 IEEE International Conference on Multimedia and Expo (ICME).

[41]  Xiaobo Lu,et al.  WeSamBE: A Weight-Sample-Based Method for Background Subtraction , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[42]  Lucia Maddalena,et al.  Extensive Benchmark and Survey of Modeling Methods for Scene Background Initialization , 2017, IEEE Transactions on Image Processing.

[43]  Guoying Zhao,et al.  Background Subtraction Using Spatio-Temporal Group Sparsity Recovery , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[44]  Guoying Zhao,et al.  Background Subtraction Based on Low-Rank and Structured Sparse Decomposition , 2015, IEEE Transactions on Image Processing.

[45]  Bo Zhang,et al.  Max-Margin Deep Generative Models for (Semi-)Supervised Learning , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Z. Zivkovic Improved adaptive Gaussian mixture model for background subtraction , 2004, ICPR 2004.

[47]  Fatih Murat Porikli,et al.  CDnet 2014: An Expanded Change Detection Benchmark Dataset , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[48]  Marc Van Droogenbroeck,et al.  ViBE: A powerful random technique to estimate the background in video sequences , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[49]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[50]  Michael Jong Kim,et al.  Thompson Sampling for Stochastic Control: The Finite Parameter Case , 2017, IEEE Transactions on Automatic Control.