Learning to Model Spatial Dependency: Semi-Supervised Discriminative Random Fields

We present a novel, semi-supervised approach to training discriminative random fields (DRFs) that efficiently exploits labeled and unlabeled training data to achieve improved accuracy in a variety of image processing tasks. We formulate DRF training as a form of MAP estimation that combines conditional loglikelihood on labeled data, given a data-dependent prior, with a conditional entropy regularizer defined on unlabeled data. Although the training objective is no longer concave, we develop an efficient local optimization procedure that produces classifiers that are more accurate than ones based on standard supervised DRF training. We then apply our semi-supervised approach to train DRFs to segment both synthetic and real data sets, and demonstrate significant improvements over supervised DRFs in each case.

[1]  Kai-Kuang Ma,et al.  Tumor segmentation from magnetic resonance imaging by learning via one-class support vector machine , 2004 .

[2]  Martial Hebert,et al.  Discriminative random fields: a discriminative framework for contextual interaction in classification , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[3]  Chi-Hoon Lee,et al.  Efficient Spatial Classification Using Decoupled Conditional Random Fields , 2006, PKDD.

[4]  Trevor Darrell,et al.  Conditional Random Fields for Object Recognition , 2004, NIPS.

[5]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[6]  Martial Hebert,et al.  Discriminative Fields for Modeling Spatial Dependencies in Natural Images , 2003, NIPS.

[7]  Adrian Corduneanu,et al.  Data-Dependent Regularization , 2006, Semi-Supervised Learning.

[8]  G. Celeux,et al.  A Classification EM algorithm for clustering and two stochastic versions , 1992 .

[9]  W. Freeman,et al.  Generalized Belief Propagation , 2000, NIPS.

[10]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[11]  Yoshua Bengio,et al.  Semi-supervised Learning by Entropy Minimization , 2004, CAP.

[12]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[13]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[14]  Bernhard Schölkopf,et al.  Learning from labeled and unlabeled data on a directed graph , 2005, ICML.

[15]  Stephen J. Roberts,et al.  Maximum certainty data partitioning , 2000, Pattern Recognit..

[16]  Sebastian Thrun,et al.  Text Classification from Labeled and Unlabeled Documents using EM , 2000, Machine Learning.

[17]  J. Besag.  On the Statistical Analysis of Dirty Pictures , 1986 .

[18]  Mikhail Belkin,et al.  Maximum Margin Semi-Supervised Learning for Structured Variables , 2005, NIPS 2005.

[19]  José Alí Moreno,et al.  Kernel Based Method for Segmentation and Modeling of Magnetic Resonance Images , 2004, IBERAMIA.

[20]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[21]  Dale Schuurmans,et al.  Semi-Supervised Conditional Random Fields for Improved Sequence Segmentation and Labeling , 2006, ACL.

[22]  Antonio Torralba,et al.  Contextual Models for Object Detection Using Boosted Random Fields , 2004, NIPS.

[23]  Cristina Garc.  Kernel Based Method for Segmentation and Modeling of Magnetic Resonance Images , 2004 .

[24]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[25]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[26]  Mark W. Schmidt,et al.  Accelerated training of conditional random fields with stochastic gradient methods , 2006, ICML.