A Study on the Cardinality of Ordered Average Pooling in Visual Recognition

Bag-of-Words methods can be robust to image scaling, translation, and occlusion. An important step in this methodology, and other visual recognition systems like Convolutional Neural Networks, is spatial pooling, where the descriptors of neighbouring elements are combined into a local or a global feature vector. The combined vector must contain relevant information, while removing irrelevant and confusing details. Maximum and average are the most common aggregation functions used in the pooling step. In this work we present a study about the cardinality of ordered average pooling, i.e. the number of ordered elements to be aggregated such that after the pooling process the relevant information is maintained without degrading their discriminative power for classification. We provide an extensive evaluation that shows that for different values of cardinalities we can obtain results better than simple average pooling and than maximum pooling when dealing with small dictionary sizes.