Approximate constraint generation for efficient structured boosting

We propose efficient training methods (SBoost) for totally-corrective boosting based structured learning. The optimization of boosting method for structured learning is more challenging than the structured support vector machine. Basically, we propose smooth and convex formulation for boosting based structured learning, and develop approximate constraint generation together with column generation to solve the optimization with large number of constraints and variables. Because of the convexity and smoothness, the optimization in each generation iteration can be solved efficiently. We demonstrate some structured learning applications in computer vision using SBoost, including invariance learning for digit recognition, object detection and hierarchical image classification.

[1]  Jorge Nocedal,et al.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization , 1997, TOMS.

[2]  Matthew B. Blaschko,et al.  Learning equivariant structured output SVM regressors , 2011, 2011 International Conference on Computer Vision.

[3]  Chunhua Shen,et al.  A direct formulation for totally-corrective multi-class boosting , 2011, CVPR 2011.

[4]  Thomas Hofmann,et al.  Support vector machine learning for interdependent and structured output spaces , 2004, ICML.

[5]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[6]  Olivier Chapelle,et al.  Training a Support Vector Machine in the Primal , 2007, Neural Computation.

[7]  Chunhua Shen,et al.  On the Dual Formulation of Boosting Algorithms , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Koby Crammer,et al.  On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines , 2002, J. Mach. Learn. Res..

[9]  Derek Hoiem,et al.  Learning CRFs Using Graph Cuts , 2008, ECCV.

[10]  Alexander J. Smola,et al.  Convex Learning with Invariances , 2007, NIPS.

[11]  Yoram Singer,et al.  Improved Boosting Algorithms Using Confidence-rated Predictions , 1998, COLT' 98.

[12]  Jason Weston,et al.  Multi-Class Support Vector Machines , 1998 .

[13]  Daphne Koller,et al.  Learning Spatial Context: Using Stuff to Find Things , 2008, ECCV.

[14]  Christoph H. Lampert,et al.  Learning to Localize Objects with Structured Output Regression , 2008, ECCV.

[15]  Krista A. Ehinger,et al.  SUN database: Large-scale scene recognition from abbey to zoo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Venkatesan Guruswami,et al.  Multiclass learning, boosting, and error-correcting codes , 1999, COLT '99.

[17]  Ayhan Demiriz,et al.  Linear Programming Boosting via Column Generation , 2002, Machine Learning.