A Simple High Performance Approach to Semantic Segmentation

We propose a simple approach to semantic image segmentation. Our system scores low-level patches according to their class re levance, propagates these posterior probabilities to pixels and uses low- level segmentation to guide the semantic segmentation. The two main contributions of this paper are as follows. First, for the patch scoring, we describe each patch with a high-level descriptor based on the Fisher kernel and use a s et of linear classifiers. While the Fisher kernel methodology was shown t o lead to high accuracy for image classification, it has not been applied to the segmentation problem. Second, we use global image classifiers to take into account the context of the objects to be segmented. If an image as a whole is unlikely to contain an object class, then the corresponding class is not considered in the segmentation pipeline. This increases the classification a ccuracy and reduces the computational cost. We will show that despite its apparent simplicity, this system provides above state-of-the-art performance on the PASCAL VOC 2007 dataset and state-of-the-art performance on the MSRC 21 dataset.