Smart Thumbnail: Automatic Image Cropping by Mining Canonical Query Objects

In this paper, we present a query-dependent thumbnailing approach for web image search. Motivated by the fact that uniform down-sampling cannot emphasize query objects while saliency-based methods may present incorrect foreground objects, we propose to employ common object discovery (COD) algorithms to mine the underlying canonical query objects from the result image collection and adopt the detected object regions of interest (ROIs) as a guide for image cropping. To make the employed COD approach more adaptive to our scenario, we enhance it by introducing text-based search rankings. We then decide for each image whether it should be cropped and determine the final cropping boundary by expanding the detected bounding box, so that the produced thumbnails are of proper appearances. The experimental results demonstrate that our method can outperform down-sampling and saliency-based methods on both object localization accuracy and general thumbnail quality.