Statistical shape knowledge in variational motion segmentation

When interpreting an image, a human observer takes into account not only the external input given by the intensity information in the image, but also internally represented knowledge. The present work is devoted to modeling such an interaction by combining in a segmentation process both low-level image cues and statistically encoded prior knowledge about the shape of expected objects. To this end, we introduce the diffusion snake as a hybrid model combining the external energy of the Mumford-Shah functional with the internal energy of the snake. In particular, we present a method called "motion competition" as an extension of the Mumford-Shah functional which aims at maximizing the homogeneity with respect to the motion estimated in each region. These purely image-based segmentation methods are extended by a shape prior, which statistically encodes a set of training silhouettes. We propose two statistical shape models of different complexity. The first one is based on the assumption that the training shapes form a Gaussian distribution in the input space, whereas the second one assumes a Gaussian distribution upon a nonlinear mapping to an appropriate feature space. This nonlinear shape prior permits to simultaneously encode in a fully unsupervised manner a fairly complex set of shapes, such as the 2D silhouettes corresponding to several 3D objects. To make the shape prior independent of translation, rotation and scaling, we propose an intrinsic alignment of the evolving contour with the training set before applying the shape prior. Numerical results demonstrate that the evolving contour is restricted to a submanifold of familiar shapes while being entirely free to translate, rotate and scale. The shape prior compensates for ambiguous, missing or misleading low-level information. It permits a segmentation of objects of interest in images which are corrupted by noise, clutter or occlusion.