Bayes Risk for Large Scale Hierarchical Top-K Image Classification

Despite numerous efforts and recent progress, image classification remains a challenging problem, where computers are still outperformed by humans. In particular, the recent trend of large-scale image classification (thousands of images, hundreds of classes, high dimensional features), made popular by the ImageNet dataset [13] has recently received growing interest. Yet, the standard evaluation protocol, which reports only the misclassification rate, fails to produce well-behaved classifiers. The inherent difficulty of large-scale datasets, causes human beings to sometimes fail at the classification task (the three classes of ImageNet “softball”, “hardball” and “professional baseball” are for instance almost indistinguishable). Even when failing, humans nevertheless always predict an output semantically similar to the correct one. A hierarchy between concepts was therefore introduced to define a inter-class distance, to penalize classifiers outputing farfetched labels, as for instance in the 2010 and 2011 editions of the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC). Another recent trend in classification protocols is to allow classifiers to output several guesses, only taking the best one into account. Also in use in ILSVRC, this leniency allows for datasets imperfections and ambiguous images. The purpose of this master thesis is twofold. In a first part, we introduce Minimum Bayes Risk prediction to solve the problem of large-scale hierarchical top-K classification. Using an approximation of a submodular score and posterior class-probabilities given by a Logistic Regression, we get significant improvements over the naive prediction. In a second part, we report a preliminary work on improving the determination of the posterior probabilities with a new classifier called the Bayes Risk Machine. We report good improvements on top-1.