A Closed-Form Solution to Natural Image Matting

Interactive digital matting, the process of extracting a foreground object from an image based on limited user input, is an important task in image and video editing. From a computer vision perspective, this task is extremely challenging because it is massively ill-posed - at each pixel we must estimate the foreground and the background colors, as well as the foreground opacity ("alpha matte") from a single color measurement. Current approaches either restrict the estimation to a small part of the image, estimating foreground and background colors based on nearby pixels where they are known, or perform iterative nonlinear estimation by alternating foreground and background color estimation with alpha estimation. In this paper, we present a closed-form solution to natural image matting. We derive a cost function from local smoothness assumptions on foreground and background colors and show that in the resulting expression, it is possible to analytically eliminate the foreground and background colors to obtain a quadratic cost function in alpha. This allows us to find the globally optimal alpha matte by solving a sparse linear system of equations. Furthermore, the closed-form formula allows us to predict the properties of the solution by analyzing the eigenvectors of a sparse matrix, closely related to matrices used in spectral image segmentation algorithms. We show that high-quality mattes for natural images may be obtained from a small amount of user input.

[1]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Carlo Tomasi,et al.  Alpha estimation in natural images , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[3]  David Salesin,et al.  A Bayesian approach to digital matting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[4]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[5]  David Salesin,et al.  Video matting of complex scenes , 2002, SIGGRAPH.

[6]  Shmuel Peleg,et al.  Multi-sensor super-resolution , 2002, Sixth IEEE Workshop on Applications of Computer Vision, 2002. (WACV 2002). Proceedings..

[7]  Antonio Torralba,et al.  Properties and applications of shape recipes , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[8]  David Salesin,et al.  Shadow matting and compositing , 2003, ACM Trans. Graph..

[9]  Dani Lischinski,et al.  Colorization using optimization , 2004, SIGGRAPH 2004.

[10]  Allan D. Jepson,et al.  Hierarchical Eigensolver for Transition Matrices in Spectral Methods , 2004, NIPS.

[11]  Jian Sun,et al.  Lazy snapping , 2004, SIGGRAPH 2004.

[12]  Vladimir Kolmogorov,et al.  "GrabCut": interactive foreground extraction using iterated graph cuts , 2004, ACM Trans. Graph..

[13]  Jiaya Jia,et al.  Poisson matting , 2004, SIGGRAPH 2004.

[14]  Andrew Fitzgibbon,et al.  Bayesian video matting using learnt image priors , 2004, CVPR 2004.

[15]  M. Werman,et al.  Color lines: image specific color representation , 2004, CVPR 2004.

[16]  Michael F. Cohen,et al.  An iterative optimization approach for unified image segmentation and matting , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[17]  Rüdiger Westermann,et al.  RANDOM WALKS FOR INTERACTIVE ALPHA-MATTING , 2005 .

[18]  R. Szeliski Locally adapted hierarchical basis preconditioning , 2006, SIGGRAPH 2006.