Higher Order Energies for Image Segmentation

A novel energy minimization method for general higher order binary energy functions is proposed in this paper. We first relax a discrete higher order function to a continuous one, and use the Taylor expansion to obtain an approximate lower order function, which is optimized by the quadratic pseudo-Boolean optimization or other discrete optimizers. The minimum solution of this lower order function is then used as a new local point, where we expand the original higher order energy function again. Our algorithm does not restrict to any specific form of the higher order binary function or bring in extra auxiliary variables. For concreteness, we show an application of segmentation with the appearance entropy, which is efficiently solved by our method. Experimental results demonstrate that our method outperforms the state-of-the-art methods.

[1]  Ruigang Yang,et al.  Saliency-Aware Video Object Segmentation , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Xiao Liu,et al.  Random Geometric Prior Forest for Multiclass Object Segmentation , 2015, IEEE Transactions on Image Processing.

[3]  Andrew C. Gallagher,et al.  Inference for order reduction in Markov random fields , 2011, CVPR 2011.

[4]  Leo Grady,et al.  A Seeded Image Segmentation Framework Unifying Graph Cuts And Random Walker Which Yields A New Algorithm , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[5]  Xuelong Li,et al.  Robust Video Object Cosegmentation , 2015, IEEE Transactions on Image Processing.

[6]  Lena Gorelick,et al.  Fast Trust Region for Segmentation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Chen Wang,et al.  A Primal-Dual Algorithm for Higher-Order Multilabel Markov Random Fields , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Fredrik Kahl,et al.  Generalized roof duality for pseudo-boolean optimization , 2011, 2011 International Conference on Computer Vision.

[9]  Andreas Krause,et al.  Efficient Minimization of Decomposable Submodular Functions , 2010, NIPS.

[10]  Pierre Hansen,et al.  Roof duality, complementation and persistency in quadratic 0–1 optimization , 1984, Math. Program..

[11]  Xuelong Li,et al.  Detection of Co-salient Objects by Looking Deep and Wide , 2016, International Journal of Computer Vision.

[12]  Vladimir Kolmogorov,et al.  Convergent Tree-Reweighted Message Passing for Energy Minimization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Fatih Murat Porikli,et al.  Saliency-aware geodesic video object segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Siam Rfview,et al.  CONVERGENCE CONDITIONS FOR ASCENT METHODS , 2016 .

[15]  Pushmeet Kohli,et al.  Minimizing sparse higher order energy functions of discrete variables , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Scott Kirkpatrick,et al.  Optimization by Simmulated Annealing , 1983, Sci..

[17]  Pushmeet Kohli,et al.  Markov Random Fields for Vision and Image Processing , 2011 .

[18]  藤重 悟.  Submodular functions and optimization , 1991 .

[19]  Vladimir Kolmogorov,et al.  Joint optimization of segmentation and appearance models , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[20]  Jeff A. Bilmes,et al.  Submodularity beyond submodular energies: Coupling edges in graph cuts , 2011, CVPR 2011.

[21]  Pushmeet Kohli,et al.  Robust Higher Order Potentials for Enforcing Label Consistency , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Lena Gorelick,et al.  Auxiliary Cuts for General Classes of Higher Order Functionals , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Pushmeet Kohli,et al.  A Principled Deep Random Field Model for Image Segmentation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[25]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[26]  Ismail Ben Ayed,et al.  Pseudo-bound Optimization for Binary Energies , 2014, ECCV.

[27]  Vladimir Kolmogorov,et al.  What energy functions can be minimized via graph cuts? , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Ling Shao,et al.  Visual Tracking Using Strong Classifier and Structural Local Sparse Descriptors , 2015, IEEE Transactions on Multimedia.

[29]  Lena Gorelick,et al.  Submodularization for Binary Pairwise Energies , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Vladimir Kolmogorov,et al.  A New Look at Reweighted Message Passing , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Ling Shao,et al.  Visual Tracking Under Motion Blur , 2016, IEEE Transactions on Image Processing.

[32]  Ling Shao,et al.  Correspondence Driven Saliency Transfer , 2016, IEEE Transactions on Image Processing.

[33]  Yair Weiss,et al.  Globally optimal solutions for energy minimization in stereo vision using reweighted belief propagation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[34]  Wenguan Wang,et al.  Occlusion-Aware Real-Time Object Tracking , 2017, IEEE Transactions on Multimedia.

[35]  Ling Shao,et al.  Generalized Pooling for Robust Object Tracking , 2016, IEEE Transactions on Image Processing.

[36]  P. Wolfe.  Convergence Conditions for Ascent Methods. II , 1969 .

[37]  Vladimir Kolmogorov,et al.  Optimizing Binary MRFs via Extended Roof Duality , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Ling Shao,et al.  Cosaliency Detection Based on Intrasaliency Prior Transfer and Deep Intersaliency Mining , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[40]  Xuelong Li,et al.  High-Order Energies for Stereo Segmentation , 2016, IEEE Transactions on Cybernetics.

[41]  Hiroshi Ishikawa,et al.  Higher-Order Clique Reduction without Auxiliary Variables , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Pushmeet Kohli,et al.  Exact inference in multi-label CRFs with higher order cliques , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Hiroshi Ishikawa,et al.  Higher-order clique reduction in binary graph cut , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Endre Boros,et al.  A graph cut algorithm for higher-order Markov Random Fields , 2011, 2011 International Conference on Computer Vision.

[45]  Xuelong Li,et al.  Improving Level Set Method for Fast Auroral Oval Segmentation , 2014, IEEE Transactions on Image Processing.

[46]  Xuelong Li,et al.  Lazy Random Walks for Superpixel Segmentation , 2014, IEEE Transactions on Image Processing.

[47]  Subhashis Banerjee,et al.  Fast Approximate Inference in Higher Order MRF-MAP Labeling Problems , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Xuelong Li,et al.  Interactive Segmentation Using Constrained Laplacian Optimization , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[49]  Olga Veksler,et al.  Minimizing Sparse High-Order Energies by Submodular Vertex-Cover , 2012, NIPS.

[50]  Kwan-Liu Ma,et al.  Stereoscopic Thumbnail Creation via Efficient Stereo Saliency Detection , 2017, IEEE Transactions on Visualization and Computer Graphics.