Adaptive spatial pooling for image classification

In this paper, we propose an adaptive spatial pooling method for enhancing the discriminability of feature representation for image classification. The core idea is to adopt a spatial distribution matrix to define how the image patches are pooled together. By formulating the pooling distribution learning and classifier training jointly, our method can extract multiple spatial layouts of arbitrary shapes rather than regular rectangular regions. By proper mathematical transformation, the distributions can be learned via a boosting-like algorithm, which improves the efficiency of learning especially for large distribution matrices. Further, our method allows category-specific pooling operations to take advantage of the different spatial layouts of different categories. Experimental results on three benchmark datasets UIUC-Sports, 21-Land-Use and Scene 15 demonstrate the effectiveness of our method. HighlightsPooling operator is parameterized as a distribution matrix.Each category enjoys its own pooling operator.The pooling distribution learning and the classifier training are jointly formulated.The problem is solved efficiently via a boosting-like algorithm.The proposed models outperform many other popular methods.