Scene Segmentation Based On Object Model Using Multisensory Information

We propose a method for model- based scene segmentation which separates object instances from the background in the scene. An object model is composed of parameterized geo- metric primitives constrained to each other accord- ing to its shape and size variations, and the rela- tionships to other object classes. As sensory data, we use a pair of color stereo images which provides color and disparity information. Using these infor- mation and the constraints in model descriptions, we approach to the problems of scene segmenta- tion and object recognition. First, we construct a height map, which shows a top view of the input scene, transforming the disparity information into 3-D space based on the height and tilt of the camera system. Using the size constraints of the object model, we separate candidate areas for object instances from the back- ground. Next, parts of the model are extracted from the scene descriptions inside each candidate area, and free parameters in the object model are determined so that the instance can be best fitted to the model. Finally, uncertainty measure for pla- nar patches is defined and applied to the extracted parts. We show the preliminary results using car models.