Background subtraction with multi-scale structured low-rank and sparse factorization

Abstract Low-rank and sparse factorization, which models the background as a low-rank matrix and the foreground as the contiguously corrupted outliers, exhibits excellent performance in background subtraction, in which the structured constraints of the foreground usually play a very essential role. In this paper, we propose a novel approach with multi-scale structured low-rank and sparse factorization for background subtraction. Different from the conventional methods that only enforce the smoothness between the spatial neighbors, we propose to explore the structured smoothness with both appearance consistency and spatial compactness in the low-rank and sparse factorization framework. Moreover, we integrate structural information at different scales into the formulation for robustness. We also design a low-rank decomposition scheme to improve the computational efficiency of the optimization algorithm. Extensive experiments on benchmark datasets GTFD and CDnet suggest that our approach achieves big superior performance against the state-of-the-art methods.