Adjusting the Ground Truth Annotations for Connectivity-Based Learning to Delineate

Deep learning-based approaches to delineating 3D structure depend on accurate annotations to train the networks. Yet, in practice, people, no matter how conscientious, have trouble precisely delineating in 3D and on a large scale, in part because the data is often hard to interpret visually and in part because the 3D interfaces are awkward to use. In this paper, we introduce a method that explicitly accounts for annotation inaccuracies. To this end, we treat the annotations as active contour models that can deform themselves while preserving their topology. This enables us to jointly train the network and correct potential errors in the original annotations. The result is an approach that boosts performance of deep networks trained with potentially inaccurate annotations.

[1]  Chao Lu,et al.  A learning based deformable template matching method for automatic rib centerline extraction and labeling in CT images , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Pascal Fua,et al.  Detecting Irregular Curvilinear Structures in Gray Scale and Color Imagery Using Multi-directional Oriented Flux , 2013, 2013 IEEE International Conference on Computer Vision.

[3]  Geoffrey E. Hinton,et al.  Machine Learning for Aerial Image Labeling , 2013 .

[4]  Pascal Fua,et al.  Joint Segmentation and Path Classification of Curvilinear Structures , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Xin Huang,et al.  Road centreline extraction from high‐resolution imagery based on multiscale structural features and support vector machines , 2009 .

[6]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[7]  Min Bai,et al.  Learning Deep Structured Active Contours End-to-End , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  Demetri Terzopoulos,et al.  Constraints on Deformable Models: Recovering 3D Shape and Nonrigid Motion , 1988, Artif. Intell..

[9]  Pascal Fua,et al.  Promoting Connectivity of Network-Like Structures by Enforcing Region Separation , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Sanja Fidler,et al.  Devil Is in the Edges: Learning Semantic Boundaries From Noisy Annotations , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[12]  Alejandro F. Frangi,et al.  Muliscale Vessel Enhancement Filtering , 1998, MICCAI.

[13]  Andrew Zisserman,et al.  Multi-task Self-Supervised Visual Learning , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[14]  Sanja Fidler,et al.  Object Instance Annotation With Deep Extreme Level Set Evolution , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[16]  Christian Heipke,et al.  Network snakes: graph-based object delineation with active contour models , 2010, Machine Vision and Applications.

[17]  Geoffrey E. Hinton,et al.  Learning to Detect Roads in High-Resolution Aerial Images , 2010, ECCV.

[18]  Luc Van Gool,et al.  Deep Retinal Image Understanding , 2016, MICCAI.

[19]  Vincent Lepetit,et al.  Multiscale Centerline Detection , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Pascal Fua,et al.  Tracing in 2D to Reduce the Annotation Effort for 3D Deep Delineation , 2018, Medical Image Anal..

[21]  J. K. Smith,et al.  Vessel tortuosity and brain tumor malignancy: a blinded study. , 2005, Academic radiology.

[22]  Zhi Zhou,et al.  TReMAP: Automatic 3D Neuron Reconstruction Based on Tracing, Reverse Mapping and Assembling of 2D Projections , 2015, Neuroinformatics.

[23]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Jan Dirk Wegner,et al.  A Higher-Order CRF Model for Road Network Extraction , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Pascal Fua,et al.  Recurrent U-Net for Resource-Constrained Segmentation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[26]  Demetri Terzopoulos,et al.  End-to-End Trainable Deep Active Contour Models for Automated Image Segmentation: Delineating Buildings in Aerial Imagery , 2020, ECCV.

[27]  Yang Zou,et al.  Simultaneous Edge Alignment and Learning , 2018, ECCV.

[28]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[29]  Pascal Fua,et al.  Model-Based Optimization: Accurate and Consistent Site Modeling , 1996 .

[30]  Shaohua Kevin Zhou,et al.  Hierarchical Discriminative Framework for Detecting Tubular Structures in 3D Images , 2013, IPMI.

[31]  Hanchuan Peng,et al.  Virtual finger boosts three-dimensional imaging and microsurgery as well as terabyte volume image visualization and analysis , 2014, Nature Communications.

[32]  Max W. K. Law,et al.  Three Dimensional Curvilinear Structure Detection Using Optimally Oriented Flux , 2008, ECCV.

[33]  Deva Ramanan,et al.  Increasing the density of Active Appearance Models , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Sanja Fidler,et al.  DARNet: Deep Active Ray Network for Building Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Raquel Urtasun,et al.  DeepRoadMapper: Extracting Road Topology from Aerial Images , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[36]  Tolga Tasdizen,et al.  Image Segmentation with Cascaded Hierarchical Models and Logistic Disjunctive Normal Networks , 2013, 2013 IEEE International Conference on Computer Vision.

[37]  Christian Heipke,et al.  EMPIRICAL EVALUATION OF AUTOMATICALLY EXTRACTED ROAD AXES , 1998 .

[38]  Pascal Fua,et al.  Model driven edge detection , 1990, Machine Vision and Applications.

[39]  Hanchuan Peng,et al.  Automatic tracing of ultra-volumes of neuronal images , 2016, Nature Methods.

[40]  Pascal Fua,et al.  Beyond the Pixel-Wise Loss for Topology-Aware Delineation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.