Fast and Optimal Laplacian Solver for Gradient-Domain Image Editing using Green Function Convolution

In computer vision, the gradient and Laplacian of an image are used in different applications, such as edge detection, feature extraction, and seamless image cloning. Computing the gradient of an image is straightforward since numerical derivatives are available in most computer vision toolboxes. However, the reverse problem is more difficult, since computing an image from its gradient requires to solve the Laplacian equation, also called Poisson equation. Current discrete methods are either slow or require heavy parallel computing. The objective of this paper is to present a novel fast and robust method of solving the image gradient or Laplacian with minimal error, which can be used for gradient domain editing. By using a single convolution based on a numerical Green's function, the whole process is faster and straightforward to implement with different computer vision libraries. It can also be optimized on a GPU using fast Fourier transforms and can easily be generalized for an n dimension image. The tests show that, for images of resolution 801x1200, the proposed GFC can solve 100 Laplacian in parallel in around 1.0 milliseconds ms. This is orders of magnitude faster than our nearest competitor which requires 294ms for a single image. Furthermore, we prove mathematically and demonstrate empirically that the proposed method is the least error solver for gradient domain editing. The developed method is also validated with examples of Poisson blending, gradient removal, and the proposed gradient domain merging GDM. Finally, we present how the GDM can be leveraged in future works for convolutional neural networks CNN.

[1]  C. Lawrence Zitnick,et al.  Fast Edge Detection Using Structured Forests , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Sofiane Achiche,et al.  Computing the Spatial Probability of Inclusion inside Partial Contours for Computer Vision Applications , 2018, ArXiv.

[3]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[4]  Li Xu,et al.  Hierarchical Image Saliency Detection on Extended CSSD , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  G. Arfken Mathematical Methods for Physicists , 1967 .

[6]  Li Xu,et al.  Hierarchical Saliency Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Nancy S. Pollard,et al.  Real-time gradient-domain painting , 2008, ACM Trans. Graph..

[8]  Pascal Barla,et al.  Diffusion curves: a vector representation for smooth-shaded images , 2008, ACM Trans. Graph..

[9]  Stefan Jeschke,et al.  A GPU Laplacian solver for diffusion curves and Poisson image editing , 2009, ACM Trans. Graph..

[10]  Michael F. Cohen,et al.  GradientShop: A gradient-domain optimization framework for image and video filtering , 2010, TOGS.

[11]  Jinhui Tang,et al.  Richer Convolutional Features for Edge Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Xiang Bai,et al.  Richer Convolutional Features for Edge Detection , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Masatoshi Okutomi,et al.  Seamless image cloning by a closed form solution of a modified Poisson problem , 2012, SA '12.

[14]  G. Arfken,et al.  Mathematical methods for physicists 6th ed. , 1996 .

[15]  Robert Laganiere,et al.  OpenCV 2 Computer Vision Application Programming Cookbook , 2011 .

[16]  Peter I. Corke,et al.  Robotics, Vision and Control - Fundamental Algorithms in MATLAB® , 2011, Springer Tracts in Advanced Robotics.

[17]  Cyril Concolato,et al.  Biharmonic diffusion curve images from boundary elements , 2013, ACM Trans. Graph..

[18]  Stephen Lin,et al.  Diffusion curve textures for resolution independent texture mapping , 2012, ACM Trans. Graph..

[19]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[20]  Timothy Sun,et al.  Fast multipole representation of diffusion curves and points , 2014, ACM Trans. Graph..

[21]  Sofiane Achiche,et al.  Novel Convolution Kernels for Computer Vision and Shape Analysis based on Electromagnetism , 2018, ArXiv.

[22]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[23]  Stanimire Tomov,et al.  Optimizing the Fast Fourier Transform Using Mixed Precision on Tensor Core Hardware , 2018, 2018 IEEE 25th International Conference on High Performance Computing Workshops (HiPCW).

[24]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.