Scale-Adaptive Superpixels

Size uniformity is one of the prominent features of superpixels. However, size uniformity rarely conforms to the varying content of an image. The chosen size of the superpixels therefore represents a compromise how to obtain the fewest superpixels without losing too much important detail. We present an image segmentation technique that generates compact clusters of pixels grown sequentially, which automatically adapt to the local texture and scale of an image. Our algorithm liberates the user from the need to choose of the right superpixel size or number. The algorithm is simple and requires just one input parameter. In addition, it is computationally very efficient, approaching real-time performance, and is easily extensible to three-dimensional image stacks and video volumes. We demonstrate that our superpixels superior to the respective state-of-the-art algorithms on quantitative benchmarks. Superpixels are a powerful preprocessing tool for image simplification. They reduce the number of image primitives from millions of pixels to a few thousands superpixels. Since their introduction [26], they have found their way into a wide-range of Computer Vision applications such as body model estimation [24], multi-class segmentation [15], depth estimation [36], object localization [14], optical flow [22], and tracking [34]. What differentiates superpixel algorithms from traditional segmentation algorithms [12, 13] are the properties of uniform size, compactness, limited adjacency, and computational efficiency [9, 18]. Despite such widespread use, the uniform size assumption of superpixels ignores the fact that real-world images do not have uniform visual complexity. Instead, such images simultaneously feature highly variable, textured regions together with more homogeneous ones. As a consequence, superpixel methods oversegment texture-less areas while under-segment textured regions. Thus, the price to pay for image-simplification using superpixels is that structures smaller than the chosen superpixel size have to be sacrificed. In this paper, we present an algorithm that alleviates the problem of the superpixel-size trade-off by relaxing one widelyimposed criterion for superpixels, namely, uniform size. Our algorithm obtains image segments that are smaller or larger in areas of high or low visual complexity, respectively, thereby achieving scale-adaptiveness (Fig. 1). We refer to the generated segments as Adaptels. Compared to the state-of-the-art, adaptels offers several advantages. There is no need to choose the size of the segments since their size evolves automatically. Similarly, the location and the number of segments are automatically chosen to conform to the image content. Our algorithm grows segments ensuring conFigure 1. The dilemma of choosing the right superpixel size retain detail and obtain too many superpixels (left image), or lose structures smaller than the superpixel size (center image)? With adaptels, the choice of size is automatic, the number of superpixels is kept small despite the advantages of conventional superpixel-based over-segmentation. nectivity from the start and thus requires no post-processing unlike some others [13, 9]. The resulting adaptels are compact, with a limited degree of adjacency. The algorithmic complexity is linear in the number of pixels and is near real-time without using any specialized hardware. Notably, the algorithm requires only a single input parameter.