Two-Stage Structure-Focused Contrastive Learning for Automatic Identification and Localization of Complex Pelvic Fractures

Pelvic fracture is a severe trauma with a high rate of morbidity and mortality. Accurate and automatic diagnosis and surgical planning of pelvic fracture require effective identification and localization of the fracture zones. This is a challenging task due to the complexity of pelvic fractures, which often exhibit multiple fragments and sites, large fragment size differences, and irregular morphology. We have developed a novel two-stage method for the automatic identification and localization of complex pelvic fractures. Our method is unique in that it allows to combine the symmetry properties of the pelvic anatomy and capture the symmetric feature differences caused by the fracture on both the left and right sides, thereby overcoming the limitations of existing methods which consider only image or geometric features. It implements supervised contrastive learning with a novel Siamese deep neural network, which consists of two weight-shared branches with a structural attention mechanism, to minimize the confusion of local complex structures of the pelvic bones with the fracture zones. A structure-focused attention (SFA) module is designed to capture the spatial structural features and enhances the recognition ability of fracture zones. Comprehensive experiments on 103 clinical CT scans from the publicly available dataset CTPelvic1K show that our method achieves a mean accuracy and sensitivity of 0.92 and 0.93, which are superior to those reported with three SOTA contrastive learning methods and five advanced classification networks, demonstrating the effectiveness of identifying and localizing various types of complex pelvic fractures from clinical CT images.

[1]  D. Beyersdorff,et al.  Supervised Contrastive Learning to Classify Paranasal Anomalies in the Maxillary Sinus , 2022, MICCAI.

[2]  J. Egger,et al.  A review on AI-based medical image computing in head and neck surgery , 2022, Physics in medicine and biology.

[3]  Jiyang Chen,et al.  Attention mechanism-based deep learning method for hairline fracture detection in hand X-rays , 2022, Neural Computing and Applications.

[4]  S. Dogramadzi,et al.  Automatic detection and classification of peri-prosthetic femur fracture , 2022, International Journal of Computer Assisted Radiology and Surgery.

[5]  Sulochana Wadhwani,et al.  Bone fractures detection using support vector machine and error backpropagation neural network , 2021, Optik.

[6]  Stephen Lin,et al.  Video Swin Transformer , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Rashedur M. Rahman,et al.  Detecting pelvic fracture on 3D-CT using deep convolutional neural networks with multi-orientated slab images , 2021, Scientific Reports.

[8]  Juan José Jiménez-Delgado,et al.  Complex fracture reduction by exact identification of the fracture zone , 2021, Medical Image Anal..

[9]  Le Lu,et al.  A scalable physician-level deep learning algorithm detects universal trauma on pelvic radiographs , 2021, Nature Communications.

[10]  Mathias Unberath,et al.  An Automated Deep Learning Method for Tile AO/OTA Pelvic Fracture Severity Grading from Trauma whole-Body CT , 2021, Journal of Digital Imaging.

[11]  Lindsey Westover,et al.  Quantitative analysis of regional specific pelvic symmetry , 2021, Medical & Biological Engineering & Computing.

[12]  Hu Han,et al.  Deep learning to segment pelvic bones: large-scale CT datasets and baseline models , 2020, International Journal of Computer Assisted Radiology and Surgery.

[13]  Franco Scarselli,et al.  Multi-Modal Siamese Network for Diagnostically Similar Lesion Retrieval in Prostate MRI , 2020, IEEE Transactions on Medical Imaging.

[14]  G Kleinszig,et al.  Fracture reduction planning and guidance in orthopaedic trauma surgery via multi-body image registration , 2020, Medical Image Anal..

[15]  Bingbing Ni,et al.  Deep-learning-assisted detection and segmentation of rib fractures from CT scans: Development and validation of FracNet , 2020, EBioMedicine.

[16]  M. Shamim Hossain,et al.  MetaCOVID: A Siamese neural network framework with contrastive loss for n-shot diagnosis of COVID-19 patients , 2020, Pattern Recognition.

[17]  D. Demetriades,et al.  Trauma Pelvic X-Ray: Not All Pelvic Fractures are Created Equally , 2020 .

[18]  R. Coimbra,et al.  Severe Pelvic Fracture in the Elderly: High Morbidity, Mortality, and Resource Utilization , 2020, The American surgeon.

[19]  Hang Joon Jo,et al.  Classification of femur fracture in pelvic X-ray images using meta-learned deep neural network , 2020, Scientific Reports.

[20]  Ziemowit Klimonda,et al.  Early Prediction of Response to Neoadjuvant Chemotherapy in Breast Cancer Sonography Using Siamese Convolutional Neural Networks , 2020, IEEE Journal of Biomedical and Health Informatics.

[21]  Adam P. Harrison,et al.  Anatomy-Aware Siamese Network: Exploiting Semantic Asymmetry for Accurate Pelvic Fracture Detection in X-ray Images , 2020, ECCV.

[22]  Yi Wang,et al.  Rectifying Supporting Regions With Mixed and Active Supervision for Rib Fracture Recognition , 2020, IEEE Transactions on Medical Imaging.

[23]  Ce Liu,et al.  Supervised Contrastive Learning , 2020, NeurIPS.

[24]  Richard D. White,et al.  Using Transfer Learning and Class Activation Maps Supporting Detection and Localization of Femoral Fractures on Anteroposterior Radiographs , 2020, 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI).

[25]  Sing Chun Lee,et al.  Exploring Partial Intrinsic and Extrinsic Symmetry in 3D Medical Imaging , 2020, Medical Image Anal..

[26]  Hao Chen,et al.  Multi-Task Deep Model With Margin Ranking Loss for Lung Nodule Analysis , 2020, IEEE Transactions on Medical Imaging.

[27]  Geoffrey E. Hinton,et al.  A Simple Framework for Contrastive Learning of Visual Representations , 2020, ICML.

[28]  Deepa Joshi,et al.  A survey of fracture detection techniques in bone X-ray images , 2020, Artificial Intelligence Review.

[29]  Mario Ceresa,et al.  Re-Identification and Growth Detection of Pulmonary Nodules without Image Registration Using 3D Siamese Neural Networks , 2019, Medical Image Anal..

[30]  Maha S. Ead,et al.  Investigation of pelvic symmetry using CAD software , 2019, Medical & Biological Engineering & Computing.

[31]  Ross B. Girshick,et al.  Momentum Contrast for Unsupervised Visual Representation Learning , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  M. Bernstein,et al.  Multidetector CT in Vascular Injuries Resulting from Pelvic Fractures: A Primer for Diagnostic Radiologists. , 2019, Radiographics : a review publication of the Radiological Society of North America, Inc.

[33]  Adam P. Harrison,et al.  Weakly Supervised Universal Fracture Detection in Pelvic X-rays , 2019, MICCAI.

[34]  Kilian Q. Weinberger,et al.  Convolutional Networks with Dense Connectivity , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Zuzanna Krawczyk,et al.  Bones detection in the pelvic area on the basis of YOLO neural network , 2018, 19th International Conference Computational Problems of Electrical Engineering.

[36]  Saeed Hassanpour,et al.  Deep neural networks for automatic detection of osteoporotic vertebral fractures on CT scans , 2018, Comput. Biol. Medicine.

[37]  Jason Yosinski,et al.  An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution , 2018, NeurIPS.

[38]  E. Aghayev,et al.  Evaluation of strategies for the treatment of type B and C pelvic fractures: RESULTS FROM THE GERMAN PELVIC INJURY REGISTER , 2018, The bone & joint journal.

[39]  K. Garala,et al.  Radiography, anatomy and imaging in pelvic fractures , 2018, Orthopaedics and Trauma.

[40]  Abhinav Gupta,et al.  Non-local Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  Alexander R. Vaccaro,et al.  Sacral Fractures and Associated Injuries , 2017, Global spine journal.

[42]  Juan José Jiménez-Delgado,et al.  Identification of fracture zones and its application in automatic bone fracture reduction , 2017, Comput. Methods Programs Biomed..

[43]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[45]  Huixiang Wang,et al.  Application of an innovative computerized virtual planning system in acetabular fracture surgery: A feasibility study. , 2016, Injury.

[46]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[48]  Jan Buschbaum,et al.  Computer-assisted fracture reduction: a new approach for repositioning femoral fractures and planning reduction paths , 2015, International Journal of Computer Assisted Radiology and Surgery.

[49]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[50]  D. Wisner,et al.  Indications and performance of pelvic radiography in patients with blunt trauma. , 2012, The American journal of emergency medicine.

[51]  Ronald M. Summers,et al.  Automated detection of pelvic fractures from volumetric CT images , 2012, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI).

[52]  Yen-Wei Chen,et al.  Computer-Assisted Preoperative Planning for Reduction of Proximal Femoral Fracture Using 3-D-CT Data , 2009, IEEE Transactions on Biomedical Engineering.

[53]  Yann LeCun,et al.  Dimensionality Reduction by Learning an Invariant Mapping , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[54]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[55]  Marvin Tile,et al.  Acute Pelvic Fractures: I. Causation and Classification , 1996, The Journal of the American Academy of Orthopaedic Surgeons.

[56]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[58]  S. K. Zhou,et al.  Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives , 2022, Medical Image Anal..

[59]  Arindam Biswas,et al.  Long-bone fracture detection in digital X-ray images based on digital-geometric techniques , 2016, Comput. Methods Programs Biomed..