CAN3D: Fast 3D Medical Image Segmentation via Compact Context Aggregation

Direct automatic segmentation of objects from 3D medical imaging, such as magnetic resonance (MR) imaging, is challenging as it often involves accurately identifying a number of individual objects with complex geometries within a large volume under investigation. To address these challenges, most deep learning approaches typically enhance their learning capability by substantially increasing the complexity or the number of trainable parameters within their models. Consequently, these models generally require long inference time on standard workstations operating clinical MR systems and are restricted to high-performance computing hardware due to their large memory requirement. Further, to fit 3D dataset through these large models using limited computer memory, trade-off techniques such as patch-wise training are often used which sacrifice the fine-scale geometric information from input images which could be clinically significant for diagnostic purposes. To address these challenges, we present a compact convolutional neural network with a shallow memory footprint to efficiently reduce the number of model parameters required for state-of-art performance. This is critical for practical employment as most clinical environments only have low-end hardware with limited computing power and memory. The proposed network can maintain data integrity by directly processing large full-size 3D input volumes with no patches required and significantly reduces the computational time required for both training and inference. We also propose a novel loss function with extra shape constraint to improve the accuracy for imbalanced classes in 3D MR images. Compared to other state-of-art approaches (U-Net3D, improved U-Net3D and V-Net), the proposed network reduced the number of parameters up to two orders of magnitude and achieve much faster inference, up to 5 times when predicting with a CPU (instead of GPU). For the open accessed OAI-ZIB knee dataset, the proposed approach achieved dice coefficient accuracy of 0.98 ± 0.00 and 0.88 ± 0.04 for femoral bone and tibial cartilage segmentation, respectively, while reducing the mean surface distance error to 0.36 ± 0.20 mm and 0.29 ± 0.10 mm when training volume-wise under only 12G VRAM. Our proposed CAN3D demonstrated high accuracy and efficiency on a pelvis 3D MR imaging dataset for prostate cancer consisting of 211 images with expert manual semantic labels (bladder, body, bone, rectum, prostate) that we have also released publicly for scientific use as part of this work1