Semisupervised manifold learning for color transfer between multiview images

In multiview image stitching, the colors of images in a scene might vary when images are taken under different illumination or camera settings. A common way to produce a seamless stitched image is to transform the colors of a target image to match that of a source image. In this paper we present a color transfer method based on two premises: first, pixels in the generated image should have similar colors with their corresponding pixels in the source image. Second, pixels with similar colors should still have similar colors after color transfer. Our method can be considered as a semisupervised manifold learning approach, where the corresponding pixels of the input images serve as the labeled data. Our goal is to learn a final image which not only shares the same colors with the source image but also has the same image structure with the target image. While manifold learning methods aim to find an embedded space to represent the data with minimum structure loss, the proposed method further constrains the solution space using the labeled data. This paper introduces a parametric linear method and a nonparametric nonlinear method to tackle different types of color changes. Experimental results show the effectiveness of our methods both quantitatively and qualitatively.

[1]  Mikhail Belkin,et al.  Semi-Supervised Learning on Riemannian Manifolds , 2004, Machine Learning.

[2]  Miguel Oliveira,et al.  Unsupervised local color correction for coarsely registered images , 2011, CVPR 2011.

[3]  B. Funt,et al.  Diagonal versus affine transformations for color correction. , 2000, Journal of the Optical Society of America. A, Optics, image science, and vision.

[4]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[5]  Stephen Lin,et al.  A New In-Camera Imaging Model for Color Computer Vision and Its Application , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[7]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[8]  Chi-Keung Tang,et al.  Image registration with global and local luminance alignment , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[9]  Chi-Keung Tang,et al.  Local color transfer via probabilistic segmentation by expectation-maximization , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Yuan Yan Tang,et al.  A Manifold Alignment Approach for Hyperspectral Image Visualization With Natural Color , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Dani Lischinski,et al.  Colorization using optimization , 2004, ACM Trans. Graph..

[12]  Jun Zhou,et al.  Manifold alignment based color transfer for multiview image stitching , 2013, 2013 IEEE International Conference on Image Processing.

[13]  André Kaup,et al.  Histogram-Based Prefiltering for Luminance and Chrominance Compensation of Multiview Video , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  François Pitié,et al.  Automated colour grading using colour distribution transfer , 2007, Comput. Vis. Image Underst..

[15]  Youngbae Hwang,et al.  Color Transfer Using Probabilistic Moving Least Squares , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Klaus Mueller,et al.  Transferring color to greyscale images , 2002, ACM Trans. Graph..

[17]  D H Brainard,et al.  Bayesian color constancy. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[18]  Wei Xu,et al.  Performance evaluation of color correction approaches for automatic multi-view image and video stitching , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.