Automatic calcaneus fracture identification and segmentation using a multi-task U-Net

Calcaneus is the bone in the foot that bears most of the body weight and calcaneus fracture is the most common type of tarsal bone fractures. Plain radiograph examination is usually the first step of calcaneus fracture diagnosis because of its convenience and low cost. A multi-task U-Net is proposed in this paper to develop a computer aided calcaneus fracture diagnosis system. Our approach is an end-to-end CNN for identification and segmentation of calcaneus fracture, which uses regularization of the two tasks for mutual performance enhancement. First, a novel radiograph normalization method to obtain scale rotation invariance under different monochrome type is employed. Second, a classification header with feature from decoder and encoder is added to U-Net for multitask. Finally, a conditional dice-loss which can promote model performance under rough-ground-truth supervision is adopted in training. Experiments show that the network predicts fracture regions more precise than the rough ground-truth and identifies fracture with sensitivity of 99.53% and specificity of 98.59%.

[1]  Hengshuang Zhao,et al.  GridMask Data Augmentation , 2020, ArXiv.

[2]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Ronald M. Summers,et al.  Deep convolutional networks for automated detection of posterior-element fractures on spine CT , 2016, SPIE Medical Imaging.

[4]  Kayvan Najarian,et al.  Fracture Detection in Traumatic Pelvic CT Images , 2012, Int. J. Biomed. Imaging.

[5]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[6]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Horst Bischof,et al.  Global localization of 3D anatomical structures by pre-filtered Hough Forests and discrete optimization , 2013, Medical Image Anal..

[8]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[9]  Sébastien Ourselin,et al.  Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations , 2017, DLMIA/ML-CDS@MICCAI.

[10]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[11]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Kenneth Badillo,et al.  Multidetector CT evaluation of calcaneal fractures. , 2011, Radiographics : a review publication of the Radiological Society of North America, Inc.

[13]  Huiqi Li,et al.  Automatic analysis system of calcaneus radiograph: Rotation-invariant landmark detection for calcaneal angle measurement, fracture identification and fracture region segmentation , 2019, Comput. Methods Programs Biomed..

[14]  Marios Anthimopoulos,et al.  Lung Pattern Classification for Interstitial Lung Diseases Using a Deep Convolutional Neural Network , 2016, IEEE Transactions on Medical Imaging.

[15]  Wen-June Wang,et al.  Real-Time Automated Segmentation and Classification of Calcaneal Fractures in CT Images , 2019, Applied Sciences.

[16]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[18]  Jia-Ching Wang,et al.  Deep learning and SURF for automated classification and detection of calcaneus fractures in CT images , 2019, Comput. Methods Programs Biomed..

[19]  Hee Chan Kim,et al.  Computer-aided diagnosis of solid breast nodules: use of an artificial neural network based on multiple sonographic features , 2004, IEEE Transactions on Medical Imaging.

[20]  B. Olsen,et al.  Calcaneal fractures: update on current treatments. , 2012, Clinics in podiatric medicine and surgery.