Boosted kernel for image categorization

Recent machine learning techniques have demonstrated their capability for identifying image categories using image features. Among these techniques, Support Vector Machines (SVM) present good results for example in Pascal Voc challenge 2011 [8], particularly when they are associated with a kernel function [28, 35]. However, nowadays image categorization task is very challenging owing to the sizes of benchmark datasets and the number of categories to be classified. In such a context, lot of effort has to be put in the design of the kernel functions and underlying semantic features. In the following of the paper we call semantic features the features describing the (semantic) content of an image. In this paper, we propose a framework to learn an effective kernel function using the Boosting paradigm to linearly combine weak kernels. We then use a SVM with this kernel to categorize image databases. More specifically, this method create embedding functions to map images in a Hilbert space where they are better classified. Furthermore, our algorithm benefits from boosting process to learn this kernel with a complexity linear with the size of the training set. Experiments are carried out on popular benchmarks and databases to show the properties and behavior of the proposed method. On the PASCAL VOC2006 database, we compare our method to simple early fusion, and on the Oxford Flowers databases we show that our method outperforms the best Multiple Kernel Learning (MKL) techniques of the literature.

[1]  Timothy J. Hazen Multi-class SVM optimization using MCE training with application to topic identification , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[2]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[3]  J. Friedman Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[4]  Alexander Zien,et al.  lp-Norm Multiple Kernel Learning , 2011, J. Mach. Learn. Res..

[5]  Paul A. Viola,et al.  Boosting Image Retrieval , 2004, International Journal of Computer Vision.

[6]  Lior Rokach,et al.  An Exploration of Research Directions in Machine Ensemble Theory and Applications , 2012, ESANN.

[7]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[8]  Manik Varma,et al.  More generality in efficient multiple kernel learning , 2009, ICML '09.

[9]  Naim Dahnoun,et al.  Studies in Computational Intelligence , 2013 .

[10]  Florent Perronnin,et al.  High-dimensional signature compression for large-scale image classification , 2011, CVPR 2011.

[11]  Sebastian Nowozin,et al.  On feature combination for multiclass object classification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[12]  Qiang Chen,et al.  Contextualizing Object Detection and Classification , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Svetlana Lazebnik,et al.  Iterative quantization: A procrustean approach to learning binary codes , 2011, CVPR 2011.

[14]  Luc Van Gool,et al.  The 2005 PASCAL Visual Object Classes Challenge , 2005, MLCW.

[15]  Matthieu Cord,et al.  Learning geometric combinations of Gaussian kernels with alternating Quasi-Newton algorithm , 2012, ESANN.

[16]  M. Kloft,et al.  l p -Norm Multiple Kernel Learning , 2011 .

[17]  Inderjit S. Dhillon,et al.  Information-theoretic metric learning , 2006, ICML '07.

[18]  Dejan Gjorgjevikj,et al.  A Multi-class SVM Classifier Utilizing Binary Decision Tree , 2009, Informatica.

[19]  Francesco Orabona,et al.  Ultra-Fast Optimization Algorithm for Sparse Multi Kernel Learning , 2011, ICML.

[20]  Motoaki Kawanabe,et al.  A procedure of adaptive kernel combination with kernel-target alignment for object classification , 2009, CIVR '09.

[21]  Matthieu Cord,et al.  Feature-based approach to semi-supervised similarity learning , 2006, Pattern Recognit..

[22]  Gunnar Rätsch,et al.  Large Scale Multiple Kernel Learning , 2006, J. Mach. Learn. Res..

[23]  Matthieu Cord,et al.  Combining visual dictionary, kernel-based similarity and learning strategy for image category retrieval , 2008, Comput. Vis. Image Underst..

[24]  Bart De Moor,et al.  Kernel-based Data Fusion for Machine Learning - Methods and Applications in Bioinformatics and Text Mining , 2009, Studies in Computational Intelligence.

[25]  Svetlana Lazebnik,et al.  Iterative quantization: A procrustean approach to learning binary codes , 2011, CVPR 2011.

[26]  Michael I. Jordan,et al.  Multiple kernel learning, conic duality, and the SMO algorithm , 2004, ICML.

[27]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[28]  Josef Kittler,et al.  Augmented Kernel Matrix vs Classifier Fusion for Object Recognition , 2011, BMVC.

[29]  V. Vapnik Estimation of Dependences Based on Empirical Data , 2006 .

[30]  Andrew Zisserman,et al.  Automated Flower Classification over a Large Number of Classes , 2008, 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing.

[31]  Corinna Cortes,et al.  Invited talk: Can learning kernels help performance? , 2009, International Conference on Machine Learning.

[32]  Frédéric Precioso,et al.  Incremental kernel learning for active image retrieval without global dictionaries , 2011, Pattern Recognit..

[33]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[34]  N. Cristianini,et al.  On Kernel-Target Alignment , 2001, NIPS.

[35]  Koby Crammer,et al.  Kernel Design Using Boosting , 2002, NIPS.

[36]  Mehryar Mohri,et al.  Two-Stage Learning Kernel Algorithms , 2010, ICML.