Local and nonlocal constraints for compressed sensing video and multi-view image recovery

Abstract In compressed sensing (CS) signal recovery tasks, employing prior constraints to exploit the inherent properties in the signal is critical to the recovery performance. Recently, multiple constraints were employed to improve the reconstruction for video sequence and multi-view image CS reconstruction since they can exploit intra- and inter-image properties simultaneously. However, correlation and dependencies among constraints were not taken into account in existing methods. This drawback prevents the multi-constraint technique from improving the reconstruction performance further. In this paper, we propose to employ local and nonlocal constraints in the CS recovery model to reconstruct video sequences and multi-view images. In the proposed CS recovery model, a pixel-wise total variation constraint is utilized to exploit the spatial local piecewise smoothness in the image. Meanwhile, a patch-based low-rank constraint is applied for the nonlocal inter-image similarity property. These two types of constraints are proposed to be applied in different dimensions with different granularities so as to alleviate the dependency and correlation between the constraints and exploit the intra- and inter-image features in images sufficiently. Besides, an efficient augmented Lagrange multipliers and alternating direction method is presented to solve the proposed local and nonlocal constraint-based CS recovery model (LNLC-CS). Experiments on test video sequences and multi-view images at various sampling rates have demonstrated the great advantages of applying independent constraints in different granularities and dimensions. The proposed method can outperform the existing state-of-the-art CS recovery approaches a lot in both the objective and the subjective quality of the recovered image. Experiments also show the proposed method has made a remarkable improvement over other competing constraint-based methods in the convergence speed and reconstruction time.