No spare parts: Sharing part detectors for image categorization

We establish that three part types are relevant for image categorization, which are all naturally shared between categories when learning a part representation for image categorization.We present an algorithm for part selection, part sharing, and image categorization by extending the AdaBoost optimization.We extend our joint optimization to a fusion with global image representations.We further improve over deep convolutional networks for image categorization. This work aims for image categorization by learning a representation of discriminative parts. Different from most existing part-based methods, we argue that parts are naturally shared between image categories and should be modeled as such. We motivate our approach with a quantitative and qualitative analysis by backtracking where selected parts come from. Our analysis shows that in addition to the category parts defining the category, the parts coming from the background context and parts from other image categories improve categorization performance. Part selection should not be done separately for each category, but instead be shared and optimized over all categories. To incorporate part sharing between categories, we present an algorithm based on AdaBoost to optimize part sharing and selection, as well as fusion with the global image representation. With a single algorithm and without the need for task-specific optimization, we achieve results competitive to the state-of-the-art on object, scene, and action categories, further improving over deep convolutional neural networks and alternative part representations.