Interactive segmentation based on iterative learning for multiple-feature fusion

This paper proposes a novel interactive segmentation method based on conditional random field (CRF) model to utilize the location and color information contained in user input. The CRF is configured with the optimal weights between two features, which are the color Gaussian Mixture Model (GMM) and probability model of location information. To construct the CRF model, we propose a method to collect samples for the cuttraining tasks of learning the optimal weights on a single image's basis and updating the parameters of features. To refine the segmentation results iteratively, our method applies the active learning strategy to guide the process of CRF model updating or guide users to input minimal training data for training the optimal weights and updating the parameters of features. Experimental results show that the proposed method demonstrates qualitative and quantitative improvement compared with the state-of-the-art interactive segmentation methods. The proposed method is also a convenient tool for interactive object segmentation. HighlightsAn iterative interactive segmentation method is proposed.A strategy for learning CRF parameters on a single image's basis.A strategy for updating the parameters of features iteratively.The CRF model is updated adaptively using active learning strategy.Users are guided to input new scribbles to improve the segmentation accuracy.

[1]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  RotherCarsten,et al.  TextonBoost for Image Understanding , 2009 .

[3]  Yizhou Yu,et al.  Learning image-specific parameters for interactive segmentation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Marc Toussaint,et al.  Multi-class image segmentation using conditional random fields and global classification , 2009, ICML '09.

[5]  Guilherme Hoefel Learning a two-stage SVM/CRF sequence classifier , 2008, CIKM '08.

[6]  Heung-Yeung Shum,et al.  Paint selection , 2009, SIGGRAPH 2009.

[7]  Andrew McCallum,et al.  Piecewise Training for Undirected Models , 2005, UAI.

[8]  Guillermo Sapiro,et al.  A Geodesic Framework for Fast Interactive Image and Video Segmentation and Matting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[9]  Xiangyang Wang,et al.  Color image segmentation using automatic pixel classification with support vector machine , 2011, Neurocomputing.

[10]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Jiang Wu,et al.  Interactive object extraction by merging regions with k-global maximal similarity , 2013, Neurocomputing.

[12]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[13]  Patrick Pérez,et al.  Interactive Image Segmentation Using an Adaptive GMMRF Model , 2004, ECCV.

[14]  Tong Zhang,et al.  Active learning using adaptive resampling , 2000, KDD '00.

[15]  Antonio Criminisi,et al.  TextonBoost for Image Understanding: Multi-Class Object Recognition and Segmentation by Jointly Modeling Texture, Layout, and Context , 2007, International Journal of Computer Vision.

[16]  Guillermo Sapiro,et al.  Geodesic Matting: A Framework for Fast Interactive Image and Video Segmentation and Matting , 2009, International Journal of Computer Vision.

[17]  Jean Ponce,et al.  Segmentation by transduction , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[19]  Sang Uk Lee,et al.  Nonparametric higher-order learning for interactive segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Cristian Sminchisescu,et al.  Image segmentation by figure-ground composition into maximal cliques , 2011, 2011 International Conference on Computer Vision.

[21]  Scott Cohen,et al.  Geodesic graph cut for interactive image segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Feiping Nie,et al.  Interactive Image Segmentation With Multiple Linear Reconstructions in Windows , 2011, IEEE Transactions on Multimedia.

[23]  Andrew Blake,et al.  GeoS: Geodesic Image Segmentation , 2008, ECCV.

[24]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[25]  R. Nevatia,et al.  Simultaneous Object Detection and Segmentation by Boosting Local Shape Feature based Classifier , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Camille Couprie,et al.  Power Watershed: A Unifying Graph-Based Optimization Framework , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Derek Hoiem,et al.  Learning CRFs Using Graph Cuts , 2008, ECCV.

[28]  Leo Grady,et al.  A Seeded Image Segmentation Framework Unifying Graph Cuts And Random Walker Which Yields A New Algorithm , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[29]  Andrew Blake,et al.  Geodesic star convexity for interactive image segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  H. Sebastian Seung,et al.  Query by committee , 1992, COLT '92.

[31]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[32]  Xiangjian He,et al.  Learning geodesic CRF model for image segmentation , 2012, 2012 19th IEEE International Conference on Image Processing.

[33]  Cristian Sminchisescu,et al.  Constrained parametric min-cuts for automatic object segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[35]  Jitendra Malik,et al.  Normalized Cuts and Image Segmentation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Vladlen Koltun,et al.  Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials , 2011, NIPS.

[37]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Andrew Zisserman,et al.  Pylon Model for Semantic Segmentation , 2011, NIPS.

[39]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Different Scenes , 2008, ECCV.

[40]  Mark W. Schmidt,et al.  Segmenting Brain Tumors with Conditional Random Fields and Support Vector Machines , 2005, CVBIA.