Fully automatic alpha matte extraction using artificial neural networks

The alpha matte is a two-dimensional map that is used to combine two images, one containing a foreground and the other containing a background. Alpha matte extraction is performed on green-screen images and requires user interaction to tune parameters in different preprocessing and postprocessing stages to refine an alpha matte. This paper tackles the problem of fully automatic extraction of the foreground on green-screen images with extraction of the corresponding alpha matte. The method is based on a multi-layer perceptron that assigns an alpha value, from a discrete set of ten alpha values, to each patch on a green-screen image. The approach for assigning an alpha value to an image patch is based on a set of features that enhance discrimination between foreground and background. The classifier is trained to learn to separate foreground objects from green-screen backgrounds as well as to generate the corresponding alpha matte map required for subsequent digital compositing. To test how the proposed approach handles alpha matte extraction under unsuitable conditions, a 64-image dataset was generated. The main contribution is that our method overcomes two challenges publicly posed within a dataset of green-screen image sequences, donated by Hollywood Camera Work LLC . Tests with this dataset generate high-quality visual results for those two cases. These results are confirmed by comparing the proposed fully automatic alpha matte extraction with that based on the use of Adobe After Effects Creative Cloud , an application which heavily depends on user interaction.

[1]  Carsten Rother,et al.  Improving Color Modeling for Alpha Matting , 2008, BMVC.

[2]  Thanathorn Phoka,et al.  Fine tuning for green screen matting , 2017, 2017 9th International Conference on Knowledge and Smart Technology (KST).

[3]  Roberto Rosas-Romero,et al.  Learning matte extraction in green screen images with MLP classifiers and the back-propagation algorithm , 2016, 2016 International Conference on Electronics, Communications and Computers (CONIELECOMP).

[4]  In-So Kweon,et al.  Natural Image Matting Using Deep Convolutional Neural Networks , 2016, ECCV.

[5]  Carsten Rother,et al.  New appearance models for natural image matting , 2009, CVPR.

[6]  Jue Wang.  Image Matting with Transductive Inference , 2011, MIRAGE.

[7]  Xiaowu Chen,et al.  Divide and Conquer: A Self-Adaptive Approach for High-Resolution Image Matting , 2016, 2016 International Conference on Virtual Reality and Visualization (ICVRV).

[8]  Eric Dubois,et al.  Matting with Sequential Pair Selection Using Graph Transduction , 2016, VMV.

[9]  Guillermo Sapiro,et al.  A Geodesic Framework for Fast Interactive Image and Video Segmentation and Matting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[10]  Jiaya Jia,et al.  Poisson matting , 2004, SIGGRAPH 2004.

[11]  Deepu Rajan,et al.  Improving Image Matting Using Comprehensive Sampling Sets , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Jian Sun,et al.  A global sampling method for alpha matting , 2011, CVPR 2011.

[13]  Jiaya Jia,et al.  Deep Automatic Portrait Matting , 2016, ECCV.

[14]  Xiao Chen,et al.  A propagation matting method based on the Local Sampling and KNN Classification with adaptive feature space , 2016, ArXiv.

[15]  Ning Xu,et al.  Deep Image Matting , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Michael F. Cohen,et al.  An iterative optimization approach for unified image segmentation and matting , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[17]  Dani Lischinski,et al.  Spectral Matting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Manuel Menezes de Oliveira Neto,et al.  Shared Sampling for Real‐Time Alpha Matting , 2010, Comput. Graph. Forum.

[19]  Carlo Tomasi,et al.  Alpha estimation in natural images , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[20]  Jian Sun,et al.  Fast matting using large kernel matting Laplacian matrices , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Michael F. Cohen,et al.  Image and Video Matting: A Survey , 2007, Found. Trends Comput. Graph. Vis..

[22]  David Salesin,et al.  A Bayesian approach to digital matting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[23]  Jian Sun,et al.  Poisson matting , 2004, ACM Trans. Graph..

[24]  Shaohui Liu,et al.  A Comprehensive Survey on Sampling‐Based Image Matting , 2017, Comput. Graph. Forum.

[25]  P. Chitra,et al.  Image Matting Based on Weighted Color and Texture Sample Selection , 2015 .

[26]  Deepu Rajan,et al.  Sparse codes as Alpha Matte , 2014, BMVC.

[27]  Günther Palm,et al.  Learning convolutional neural networks from few samples , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[28]  Ping Wang,et al.  Adaptive propagation matting based on transparency of image , 2017, Multimedia Tools and Applications.

[29]  Chi-Keung Tang,et al.  KNN Matting , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2008 .