Segmentation via fusion of edge and needle map

This paper presents an integrated image segmentation method using edge and needle map which compensates deficiencies of using either edge-based approach or region-based approach. Segmentation of an image is the first and most difficult step toward symbolic transformation of a raw image, which is essential in image understanding. In industrial applications, the task is further complicated by the ubiquitous presence of specularity in most industrial parts. Three images taken from three different illumination directions were used to separate specular and Lambertian components in the images. Needle map is generated from Lambertian component images using photometric stereo technique. In one channel, edges are extracted and linked from the averaged Lambertian images providing one source of segmentation. The other channel, Gaussian curvature and mean curvature values are estimated at each pixel from least square local surface fit of needle map. Labeled surface type image is then generated using the signs of Gaussian and mean curvatures, where one of ten surface types is assigned to each pixel. Connected regions of identical surface type pixels provide the first level grouping, a rough initial segmentation. Edge information and initial segmentation of surface type are fed to an integration module which interprets the edges and regions in a consistent way. During interpretation regions are merged or split, edges are discarded or generated depending upon global surface fit error and consistency with neighboring regions. The output of integrated segmentation is an explicit description of surface type and contours of each region which facilitates recognition, localization and attitude determination of objects in the image.