3D segmentation of multiview images based on disparity estimation

This paper presents new methods for partitioning a set of multi-view images into 3-D regions corresponding to objects in the scene in order to parse raw multi-view data into a 3-D region based structured representation. For this purpose, color, position, and disparity information at each pixel are incorporated as an attributes vector into the segmentation process. We propose three methods, all of which are based on K-means clustering algorithm. The first method is sensitive to the estimation error of disparity at each pixel, as it is formulated assuming that the estimated disparity is accurate. We solve this problem in the second method by prohibiting estimated disparity from being used for calculating the distance between attributes vectors. Finally, a third method is proposed to reduce the calculation cost of the segmentation process. As each 3-D region has one-to-one correspondence to an object or surface in the scene, 3-D region based structured representation of multi-view images is useful and powerful for data compression, view interpolation, structure recovery, and so on. The experimental results show the potential applicability of the method to the next-generation 3-D image communication system.