论文信息 - Exponentiated Gradient Algorithms for Large-margin Structured Classification

Exponentiated Gradient Algorithms for Large-margin Structured Classification

We consider the problem of structured classification, where the task is to predict a label y from an input x, and y has meaningful internal structure. Our framework includes supervised training of Markov random fields and weighted context-free grammars as special cases. We describe an algorithm that solves the large-margin optimization problem defined in [12], using an exponential-family (Gibbs distribution) representation of structured objects. The algorithm is efficient—even in cases where the number of labels y is exponential in size—provided that certain expectations under Gibbs distributions can be calculated efficiently. The method for structured labels relies on a more general result, specifically the application of exponentiated gradient updates [7, 8] to quadratic programs.

[1] J. Lamperti. ON CONVERGENCE OF STOCHASTIC PROCESSES , 1962 .

[2] Manfred K. Warmuth,et al. Exponentiated Gradient Versus Gradient Descent for Linear Predictors , 1997, Inf. Comput..

[3] Nello Cristianini,et al. Multiplicative Updatings for Support Vector Learning , 1999 .

[4] Nello Cristianini,et al. An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[5] Michael Collins,et al. Parameter Estimation for Statistical Parsing Models: Theory and Practice of , 2001, IWPT.

[6] Koby Crammer,et al. On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines , 2002, J. Mach. Learn. Res..

[7] Andrew McCallum,et al. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[8] Ben Taskar,et al. Max-Margin Markov Networks , 2003, NIPS.

[9] Thomas Hofmann,et al. Hidden Markov Support Vector Machines , 2003, ICML.

[10] Daniel D. Lee,et al. Multiplicative Updates for Large Margin Classifiers , 2003, COLT.

[11] Manfred K. Warmuth,et al. Relative Loss Bounds for Multidimensional Regression Problems , 1997, Machine Learning.

[12] Thomas Hofmann,et al. Support vector machine learning for interdependent and structured output spaces , 2004, ICML.