Interactive Deep Editing Framework for Medical Image Segmentation

Deep neural networks exhibit superior performance in dealing with segmentation of 3D medical images. However, the accuracy of segmentation results produced by fully automatic algorithms is insufficiently high due to the complexity of medical images; as such, further manual editing is required. To solve this problem, this paper proposes an interactive editing method combined with 3D end-to-end segmentation network. In the training stage, we simulate the user interactions, which are used as training data, by comparing the segmentation automatically generated by convolutional neural network with the ground truth. User interactions are fed into the network along with the images, allowing the network to adjust the segmentation results based on user edits. Our system provides three editing tools for smartly fixing segmentation errors, which cover most commonly used editing styles in medical image segmentation. With the high-level semantic information in the network, our method can efficiently and accurately edit the 3D segmentation. The interactive editing experiments on the BraTS dataset show that our method can significantly improve the segmentation accuracy with a small number of interactions only. The proposed method presents potential for clinical applications.