Approximate Parameter Learning in Conditional Random Fields: An Empirical Investigation

We investigate maximum likelihood parameter learning in Conditional Random Fields (CRF) and present an empirical study of pseudo-likelihood (PL) based approximations of the parameter likelihood gradient. We show, as opposed to [1][2], that these parameter learning methods can be improved and evaluate the resulting performance employing different inference techniques. We show that the approximation based on penalized pseudo-likelihood (PPL) in combination with the Maximum A Posteriori (MAP) inference yields results comparable to other state of the art approaches, while providing advantages to formulating parameter learning as a convex optimization problem. Eventually, we demonstrate applicability on the task of detecting man-made structures in natural images.

[1]  J. Besag.  On the Statistical Analysis of Dirty Pictures , 1986 .

[2]  Thomas Hofmann,et al.  Learning to Model Spatial Dependency: Semi-Supervised Discriminative Random Fields , 2007 .

[3]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[4]  J. Besag.  Statistical Analysis of Non-Lattice Data , 1975 .

[5]  Mark W. Schmidt,et al.  Accelerated training of conditional random fields with stochastic gradient methods , 2006, ICML.

[6]  Filip Korc.  INTERPRETING TERRESTRIAL IMAGES OF URBAN SCENES USING DISCRIMINATIVE RANDOM FIELDS , 2008 .

[7]  Martial Hebert,et al.  Discriminative Random Fields , 2006, International Journal of Computer Vision.

[8]  Andrew McCallum,et al.  Dynamic Conditional Random Fields for Jointly Labeling Multiple Sequences , 2003 .

[9]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[10]  Federico Girosi,et al.  Parallel and Deterministic Algorithms from MRFs: Surface Reconstruction , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Martial Hebert,et al.  Exploiting Inference for Approximate Parameter Learning in Discriminative Fields: An Empirical Study , 2005, EMMCVPR.

[12]  J. Besag.  Efficiency of pseudolikelihood estimation for simple Gaussian fields , 1977 .

[13]  Andrew McCallum,et al.  Piecewise pseudolikelihood for efficient training of conditional random fields , 2007, ICML '07.

[14]  Pat Langley,et al.  Editorial: On Machine Learning , 1986, Machine Learning.

[15]  Martial Hebert,et al.  Discriminative random fields: a discriminative framework for contextual interaction in classification , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[16]  Sanjiv Kumar.  Multiclass Discriminative Fields for Parts-Based Object Detection , 2004 .

[17]  D. Greig,et al.  Exact Maximum A Posteriori Estimation for Binary Images , 1989 .

[18]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.