SIFT-flow-based color correction for multi-view video

During the multi-view video acquisition, color variation across the views tends to be incurred due to different camera positions, orientations, and local lighting conditions. Such color variation will inevitably deteriorate the performance of the follow-up multi-view video processing, such as multi-view video coding (MVC). To address this problem, an effective color correction algorithm, called the SIFT flow-based color correction (SFCC), is proposed in this paper. First, the SIFT-flow technique is used to establish point-to-point correspondences across all the views of the multi-view video. The average color is then computed based on those identified common corresponding points and used as the reference color. By minimizing the energy of the difference yielded between the color of those identified common corresponding points in each view with respect to the reference color, the color correction matrix for each view can be obtained and used to correct its color. Experimental results have shown that the proposed SFCC algorithm is able to effectively eliminate the color variation inherited in multi-view video. By further exploiting the developed SFCC algorithm as a pre-processing for the MVC, extensive simulation results have shown that the coding efficiency of the color-corrected multi-view video can be greatly improved (on average, 0.85dB, 1.27dB and 1.63dB gain for Y, U, and V components, respectively), compared with that of the original multi-view video without color correction. HighlightsPropose to exploit SIFT flow to conduct color correction for multi-view video.Use SIFT flow to identify all common corresponding points across all the views.Compute the reference color by averaging the color of the common corresponding points.Generate color correction matrix by minimizing the energy of color difference.Experimental results clearly justify the effectiveness of the proposed method.