EdgeFlow: Achieving Practical Interactive Segmentation with Edge-Guided Flow

High-quality training data play a key role in image segmentation tasks. Usually, pixel-level annotations are expensive, laborious and time-consuming for the large volume of training data. To reduce labelling cost and improve segmentation quality, interactive segmentation methods have been proposed, which provide the result with just a few clicks. However, their performance does not meet the requirements of practical segmentation tasks in terms of speed and accuracy. In this work, we propose EdgeFlow, a novel architecture that fully utilizes interactive information of user clicks with edge-guided flow. Our method achieves state-of-the-art performance without any post-processing or iterative optimization scheme. Comprehensive experiments on benchmarks also demonstrate the superiority of our method. In addition, with the proposed method, we develop an efficient interactive segmentation tool for practical data annotation tasks. The source code and tool is avaliable at https://github.com/PaddlePaddle/PaddleSeg.

[1]  Konstantin Sofiiuk,et al.  Reviving Iterative Training with Mask Guidance for Interactive Segmentation , 2021, 2022 IEEE International Conference on Image Processing (ICIP).

[2]  Marie-Pierre Jolly,et al.  Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in N-D Images , 2001, ICCV.

[3]  Alain Bretto,et al.  Random walks in directed hypergraphs and application to semi-supervised image segmentation , 2014, Comput. Vis. Image Underst..

[4]  Lavdie Rada,et al.  Image Segmentation for Intensity Inhomogeneity in Presence of High Noise , 2018, IEEE Transactions on Image Processing.

[5]  Yunchao Wei,et al.  Interactive Object Segmentation With Inside-Outside Guidance , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Kuiyuan Yang,et al.  Semantic Flow for Fast and Accurate Scene Parsing , 2020, ECCV.

[7]  Tongwei Lu,et al.  Deep Optical Flow Feature Fusion Based on 3D Convolutional Networks for Video Action Recognition , 2018, 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI).

[8]  Zhao Zhang,et al.  Interactive Image Segmentation With First Click Attention , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Zhuwen Li,et al.  Interactive Image Segmentation with Latent Diversity , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Yanjun Ma,et al.  PaddlePaddle: An Open-Source Deep Learning Platform from Industrial Practice , 2019 .

[12]  Sim Heng Ong,et al.  Regional Interactive Image Segmentation Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[13]  Junjie Bai,et al.  Error-Tolerant Scribbles Based Interactive Image Segmentation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Hyeran Byun,et al.  Localization-Aware Adaptive Pairwise Margin Loss for Fine-Grained Image Recognition , 2021, IEEE Access.

[15]  Bin Wang,et al.  Interactive active contour with kernel descriptor , 2018, Inf. Sci..

[16]  Olga Veksler,et al.  Joint Optimization of Segmentation and Color Clustering , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[17]  Bastian Leibe,et al.  Iteratively Trained Interactive Segmentation , 2018, BMVC.

[18]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Rodrigo Benenson,et al.  Large-Scale Interactive Object Segmentation With Human Annotators , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Luc Van Gool,et al.  A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Ning Xu,et al.  Deep GrabCut for Object Selection , 2017, BMVC.

[22]  Zhouchen Lin,et al.  PointFlow: Flowing Semantics Through Points for Aerial Image Segmentation , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Xilin Chen,et al.  Object-Contextual Representations for Semantic Segmentation , 2019, ECCV.

[24]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[25]  Jianfei Cai,et al.  Robust Interactive Image Segmentation Using Convex Active Contours , 2012, IEEE Transactions on Image Processing.

[26]  Luc Van Gool,et al.  Deep Extreme Cut: From Extreme Points to Object Segmentation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[27]  Xiaoci Huang,et al.  Cascade CenterNet: Robust Object Detection for Power Line Surveillance , 2021, IEEE Access.

[28]  Yuying Hao,et al.  PaddleSeg: A High-Efficient Development Toolkit for Image Segmentation , 2021, ArXiv.

[29]  Dong Liu,et al.  Deep High-Resolution Representation Learning for Human Pose Estimation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Alexandre X. Falcão,et al.  Riverbed: A Novel User-Steered Image Segmentation Method Based on Optimum Boundary Tracking , 2012, IEEE Transactions on Image Processing.

[31]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[32]  Gareth Funka-Lea,et al.  Graph Cuts and Efficient N-D Image Segmentation , 2006, International Journal of Computer Vision.

[33]  Jianping Shi,et al.  Improving Semantic Segmentation via Decoupled Body and Edge Supervision , 2020, ECCV.

[34]  Ross B. Girshick,et al.  LVIS: A Dataset for Large Vocabulary Instance Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Ling Shao,et al.  Sub-Markov Random Walk for Image Segmentation , 2016, IEEE Transactions on Image Processing.

[36]  Khairul Nizam Tahar,et al.  Homogeneous tree height derivation from tree crown delineation using Seeded Region Growing (SRG) segmentation , 2020, Geo spatial Inf. Sci..

[37]  Tao Hai,et al.  Scene segmentation based on seeded region growing for foreground detection , 2010, 2010 Sixth International Conference on Natural Computation.

[38]  Petros Maragos,et al.  Unifying the random walker algorithm and the SIR model for graph clustering and image segmentation , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[39]  Pengfei Bi,et al.  Application of Locally Invariant Robust PCA for Underwater Image Recognition , 2021, IEEE Access.

[40]  Michael Gygli,et al.  Continuous Adaptation for Interactive Object Segmentation by Learning from Corrections , 2019, ECCV.

[41]  Chang-Su Kim,et al.  Interactive Image Segmentation via Backpropagating Refinement Scheme , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Ilia Petrov,et al.  F-BRS: Rethinking Backpropagating Refinement for Interactive Segmentation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Angela Yao,et al.  Content-Aware Multi-Level Guidance for Interactive Instance Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Andrew Blake,et al.  Geodesic star convexity for interactive image segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[46]  Rolf Adams,et al.  Seeded Region Growing , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Xavier Bresson,et al.  Fast Global Minimization of the Active Contour/Snake Model , 2007, Journal of Mathematical Imaging and Vision.

[48]  Defu Lin,et al.  Toward Efficient Object Detection in Aerial Images Using Extreme Scale Metric Learning , 2021, IEEE Access.

[49]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[50]  Ning Xu,et al.  Deep Interactive Object Selection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).