Improving semantic texton forests with a Markov random field for image segmentation

Semantic image segmentation is a major and challenging problem in computer vision, which has been widely researched over decades. Recent approaches attempt to exploit contextual information at different levels to improve the segmentation results. In this paper, we propose a new approach for combining semantic texton forests (STFs) and Markov random fields (MRFs) for improving segmentation. STFs allow fast computing of texton codebooks for powerful low-level image feature description. MRFs, with the most effective algorithm in message passing for training, will smooth out the segmentation results of STFs using pairwise coherent information between neighboring pixels. We evaluate the performance of the proposed method on two well-known benchmark datasets including the 21-class MSRC dataset and the VOC 2007 dataset. The experimental results show that our method impressively improved the segmentation results of STFs. Especially, our method successfully recognizes many challenging image regions that STFs failed to do.

[1]  Antonio Criminisi,et al.  TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-class Object Recognition and Segmentation , 2006, ECCV.

[2]  Stephen Gould,et al.  Region-based Segmentation and Object Detection , 2009, NIPS.

[3]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[4]  Vladimir Kolmogorov,et al.  Convergent Tree-Reweighted Message Passing for Energy Minimization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Antonio Criminisi,et al.  Object Class Segmentation using Random Forests , 2008, BMVC.

[6]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Shimon Ullman,et al.  Class-Specific, Top-Down Segmentation , 2002, ECCV.

[8]  Stefano Soatto,et al.  Class segmentation and object localization with superpixel neighborhoods , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[9]  Huynh Thi Thanh Binh,et al.  Improving Image Segmentation Using Genetic Algorithm , 2012, 2012 11th International Conference on Machine Learning and Applications.

[10]  Pietro Perona,et al.  Object class recognition by unsupervised scale-invariant learning , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[11]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[13]  Huynh Thi Thanh Binh,et al.  Genetic algorithm in boosting for object class image segmentation , 2013, 2013 International Conference on Soft Computing and Pattern Recognition (SoCPaR).

[14]  Vladlen Koltun,et al.  Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials , 2011, NIPS.

[15]  Pushmeet Kohli,et al.  Associative hierarchical CRFs for object class image segmentation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[16]  Pedro F. Felzenszwalb,et al.  Efficient belief propagation for early vision , 2004, CVPR 2004.

[17]  Martin J. Wainwright,et al.  MAP estimation via agreement on trees: message-passing and linear programming , 2005, IEEE Transactions on Information Theory.

[18]  Marie-Pierre Jolly,et al.  Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in N-D Images , 2001, ICCV.

[19]  Sebastian Nowozin,et al.  A Comparative Study of Modern Inference Techniques for Discrete Energy Minimization Problems , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Shimon Ullman,et al.  Learning to Segment , 2004, ECCV.

[21]  Andrew Zisserman,et al.  OBJ CUT , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[22]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Martin J. Wainwright,et al.  MAP estimation via agreement on (hyper)trees: Message-passing and linear programming , 2005, ArXiv.

[24]  Zoltan Kato,et al.  A Markov random field image segmentation model for color textured images , 2006, Image Vis. Comput..

[25]  Miguel Á. Carreira-Perpiñán,et al.  Multiscale conditional random fields for image labeling , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[26]  Richard Szeliski,et al.  A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Andrew Zisserman,et al.  A Boundary-Fragment-Model for Object Detection , 2006, ECCV.

[28]  J. Besag.  On the Statistical Analysis of Dirty Pictures , 1986 .

[29]  Feng Zhu,et al.  Theme-Based Multi-class Object Recognition and Segmentation , 2010, 2010 20th International Conference on Pattern Recognition.

[30]  Shimon Ullman,et al.  Combining Top-Down and Bottom-Up Segmentation , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[31]  Stan Z. Li.  Markov Random Field Modeling in Image Analysis , 2009, Advances in Pattern Recognition.

[32]  B. Schiele,et al.  Combined Object Categorization and Segmentation With an Implicit Shape Model , 2004 .

[33]  Stephen T. Barnard,et al.  Stochastic stereo matching over scale , 1989, International Journal of Computer Vision.

[34]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[35]  Roberto Cipolla,et al.  Semantic texton forests for image categorization and segmentation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Jitendra Malik,et al.  Contour and Texture Analysis for Image Segmentation , 2001, International Journal of Computer Vision.