Comparative Analysis of Structure and Texture based Image Inpainting Techniques

There are various real world situations where, a portion of the image is lost or damaged or hidden by an unwanted object which needs an image restoration. Digital Image Inpainting is a technique which addresses such an issue. Inpainting techniques are based on interpolation, diffusion or exemplar based concepts. This paper briefly describes the application of such concepts for inpainting and provides their detailed performance analysis. It is observed that the performance of these techniques vary while restoring the structure and texture present in an image. This paper gives the limitations of each technique and suggests the choice of appropriate technique for a given scenario. Keywords—digital image inpainting, exemplar based i npainting, TV inpainting, isotropic diffusion, anisotropic diffusion. 1-INTRODUCTION A Photographic picture is a two dimensional image w hich can contain many objects. One may be intereste d in the object or scene that is hidden by another. For exam ple, a beautiful picture may contain some letters w ritten on it or a view of the Taj mahal maybe occluded or a historic painting may be torn or damaged. Here the picture b elow the letters, the occluded portion of the Taj mahal and the damaged portion of the painting needs to be res tored. This problem is addressed under various headings like di socclusion, Object Removal, Image Inpainting etc. R etrieving the information that is hidden or missing becomes diffi cult when there is no prior knowledge or reference image. Here the information surrounding the missing area and other known area has to be utilized for the restoration. Usually, the user in the form of mask specifies the unwanted foregrou nd or the object to be removed or the portion of im age to be retrieved. Clone Brush tool of Adobe Photoshop rest or the image when a sample of the image to be pla c d in the missing area is selected by the user whereas in inp ainting the missing area is automatically filled in by the algorithm. Digital image inpainting is a kind of digital image processing that modifies a portion of the image ba sed on the surrounding area in an undetectable way. The techni ques rely majorly on the diffusion and the sampling process. It has a wide variety of applications in restoration o f deteriorated photos, denoising images, creating s pecial effects in movies, digital zoom-in and edge based image compre ssion. 2. STATE OF ART The inpainting problem can be considered as assigni ng the gray levels to the missing area called as Ω with the help of gray levels in the known area Φ as shown in fig. 2.1. The boundary δΩ, between the two plays a major role in deciding the intensities in Ω. All the algorithms are iterative and try to fill in δΩ first and moves inwards successively altering δΩ each time. The algorithm stops when all the pixels in Ω are successfully assigned some values. The restoration of the structural information like edge s or textural information like repeating patterns p ose a major challenge for the inpainting techniques. Based on t he nature of filling, the algorithms could be class ified into structure based and texture based methods. 1063 Comparative Analysis of Structure and Texture based Image Inpainting Techniques ISSN-2277-1956 /V1N3-1062-1069 Figure2.1. Digital image Inpainting problem Structure based methods uses geodesic curves and the points of isophotes arriving at the boundary fo r inpainting. Isophotes are the lines joining the sam e gray levels and geodesic curves are lines followi ng the shortest possible paths between two points. When used in its pr mitive form it may result in disconnected objec ts. This is illustrated in Fig 2.2; while inpainting the black square in Fig 2.2a, a horizontal bar is expected bu t the algorithm results in two disconnected bars as in Fig 2.2b. The mathematical models for deterministic and varia tion l PDE are explained in detail in [4] and [6]. A series of Partial differential equations are used to extend i sophotes in to the missing area in [1],[2] and[11]. In [12] a convolution mask is used to extend the gray levels in to the inpainting area. The curvatures are exten ded into the inpainting area in [5]. The Texture based methods mainly rely on texture sy nthesis, [3] and [9] which grow a new image outward from an initial seed. Before a pixel is synthesized, its neighbors are sampled. Then the whole image is que ried to find out a source pixel with similar neighbors. At this point, the source pixel is copied to the pixel to be synt hesized which is the missing area. This is called as Exemplar based synt hesis. Based on whether a pixel or a sub window is used for sampling it is further classified as pixel based sa mpling and patch based sampling. The patch size, ma tching criteria and order of filling varies between algorithms. Exe mplar based inpainting is used in [7] and [15]. 3. DIGITAL IMAGE INPAINTING TECHNIQUES The digital image inpainting involves two major ste ps. First step involves the selection of area to be inpainted and the second is the inpainting algorithm which gives appropriate values for the selected area. 3.1 Inpainting area selection The area to be inpainted is selected by the user ba sed on color, region selected by user or a binary i mage specifying the missing area. Color based selection is more flexible and it could be used for specifyi ng the area irrespective of the shape, area and number of regio ns. Instead of looking for the exact color value, t he color values closer to it is also taken, into account for the qu antization effects. This method requires the missin g area to be in a unique and different color from the rest of the ima ge. The user can select the missing area through a free hand selection or polygon selection. This method is capa ble of selecting the missing area irrespective of t he color. This could be used predominantly on black and white imag es. However the missing area cannot be precisely sp cified in this method. It becomes tedious to select more than one area as in the case of imposed text on the ima ge. If the area to be inpainted remains constant across various images or the template of the damage is known, the missin g area is specified in the form of a binary image with the sa me size of the input image. This method is best sui ted to specify the black text imposed on black and white images. In pr actice the missing area is selected using any image manipulation software and given a different color which is then used for the inpainting algorithm. The user selecte d area is usually called as the mask or the region to be inpainted. 3.2 Structure based inpainting These methods are based on the Partial different ial equations which contribute to the structural in formation in an image. The differential equations which use t he concept of Interpolation, Diffusion and Total Va riational PDEs are discussed in this paper. IJECSE,Volume1,Number 3 S. Padmavathi and K. P. Soman ISSN-2277-1956 /V1N3-1062-1069 3.2.1 Interpolation Based Inpainting The simplest method uses soap film PDE, where δΩ becomes its boundary conditions. A set of linear e quations are formed with the known values of δΩ and unknown values of Ω in four major direction namely the north, south, east and west. Interpolation of the four neighbors is used to frame the equation. The δΩ forms the right hand side of the equations. The equations are solved to get the intensities of Ω. 3.2.2 Anisotropic Diffusion Based Inpainting Inpainting problem is considered as diffusion of gr ay levels from the boundary area δΩ into the unknown area Ω. The level set theory used to explain the diffusion b undaries during various periods. If the diffusion process does not depend on the direction or the presence of edges, i t is called as isotropic diffusion. Interpolation t echnique is isotropic in this sense. Anisotropic diffusion[13] is used to avoid blurring across edges. Equation 3.1 shows th e anisotropic diffusion where g represents a smooth function, K r epresents the curvature, ∇ I represent the gradient of the image and Ω represents the area other than Ω. The curvature is given by the equation 3.2.