Interleaved Object Categorization and Segmentation

This thesis is concerned with the problem of visual object categorization, that is of reeognizing unseen-before objects, localizing them in cluttered real-worid images, and assigning the correct category label. This capability is one of the core compe¬ tencies of the human visual system. Yet, Computer vision Systems are still far from reaching a comparable level of Performance.Moreover,Computer visionresearch has in the past mainly focused on the simpler and more specific problem of identifying known objects under novel viewing conditions. The visual categorization problem is closely linked to the task of figure-ground segmentation, that is of dividing the image into an object and a non-objeet part. Historically, figure-ground segmentation has often been seen as an important and even necessary preprocessing step for object recognition. However, purely bottomup approacheshave so far been unable to yield segmentationsof sufficient quality, so that most current recognition approacheshave been designed to work independently from segmentation. In contrast,this thesis considers object categorization and figure-ground segmen¬ tation as two interleaved processes that closely collaborate towards a common goal. The core part of our work is a probabilisticformulation which integrates both capabilities into a common framework. As shown in our experiments, the tight coupling between those two processes allows them to profit from each other and improve their individual Performances. The resulting approach can detect categorical objects in novel images and automatically compute a segmentationfor them. This segmenta¬ tion is then used to again improve recognition by allowing the System to focus its effort on object pixels and discard misleading influencesfrom the background. In addition to improving the recognition Performance for individual hypotheses, the top-down segmentation also allows to determine exactly from where a hypoth¬ esis draws its support. We use this information to design a hypothesis verification stage based on the MDL principle that resolves ambiguities between overlapping hypotheseson a per-pixel level and factorsout the effects of partialocclusion. Altogether, this procedureconstitutes a novel mechanismin object detection that allows to analyze scenes containing multiple objects in a principled manner. Our results show that it presents an improvement over conventional criteria based on bounding box overlap and permitsmore aecurate aeeeptancedecisions. Our approach is based on a highly flexible implicit representation for object shape that can combine the information of local parts observed on different training exam¬ ples and interpolate between the correspondingobjects. As a result, the proposed method can learn object modeis already from few training examples and achieve competitive object detection Performance with training sets that are between one and two orders of magnitude smaller than those used in comparable Systems. An extensive evaluation on several large data sets shows that the system is applicable to many different object categories, including both rigid and articulated objects.