Simultaneous super-resolution and contrast synthesis of routine clinical magnetic resonance images of the knee for improving automatic segmentation of joint cartilage: data from the Osteoarthritis Initiative.

PURPOSE High resolution 3D MR images are well suited for automated cartilage segmentation in the human knee joint. However, volumetric scans such as 3D Double-Echo Steady-State (DESS) images are not routinely acquired in clinical practice which limits opportunities for reliable cartilage segmentation using (fully) automated algorithms. In this work, a method for generating synthetic 3D MR (syn3D-DESS) images with better contrast and higher spatial resolution from routine, low resolution, 2D Turbo-Spin Echo (TSE) clinical knee scans is proposed. METHODS A UNet convolutional neural network is employed for synthesizing enhanced artificial MR images suitable for automated knee cartilage segmentation. Training of the model was performed on a large, publically available dataset from the OAI, consisting of 578 MR examinations of knee joints from 102 healthy individuals and patients with knee osteoarthritis. RESULTS The generated synthetic images have higher spatial resolution and better tissue contrast than the original 2D TSE, which allow high quality automated 3D segmentations of the cartilage. The proposed approach was evaluated on a separate set of MR images from 88 subjects with manual cartilage segmentations. It provided a significant improvement in automated segmentation of knee cartilages when using the syn3D-DESS images compared to the original 2D TSE images. CONCLUSION The proposed method can successfully synthesise 3D DESS images from 2D TSE images to provide images suitable for automated cartilage segmentation.

[1]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[2]  Christian Igel,et al.  Deep Feature Learning for Knee Cartilage Segmentation Using a Triplanar Convolutional Neural Network , 2013, MICCAI.

[3]  Stefan Zachow,et al.  Automated segmentation of knee bone and cartilage combining statistical shape knowledge and convolutional neural networks: Data from the Osteoarthritis Initiative , 2019, Medical Image Anal..

[4]  Su Ruan,et al.  Medical Image Synthesis with Context-Aware Generative Adversarial Networks , 2016, MICCAI.

[5]  Richard Kijowski,et al.  Deep convolutional neural network for segmentation of knee joint anatomy , 2018, Magnetic resonance in medicine.

[6]  J. Fripp,et al.  Automated segmentation and analysis of normal and osteoarthritic knee menisci from magnetic resonance images--data from the Osteoarthritis Initiative. , 2014, Osteoarthritis and cartilage.

[7]  Stuart Crozier,et al.  Automatic Segmentation and Quantitative Analysis of the Articular Cartilages From Magnetic Resonance Images of the Knee , 2010, IEEE Transactions on Medical Imaging.

[8]  Jurgen Fripp,et al.  Robust inverse-consistent affine CT-MR registration in MRI-assisted and MRI-alone prostate radiation therapy , 2015, Medical Image Anal..

[9]  David Dagan Feng,et al.  Synthesis of Positron Emission Tomography (PET) Images via Multi-channel Generative Adversarial Networks (GANs) , 2017, CMMI/RAMBO/SWITCH@MICCAI.

[10]  Yinghuan Shi,et al.  Ea-GANs: Edge-Aware Generative Adversarial Networks for Cross-Modality MR Image Synthesis , 2019, IEEE Transactions on Medical Imaging.

[11]  Snehashis Roy,et al.  Magnetic Resonance Image Example-Based Contrast Synthesis , 2013, IEEE Transactions on Medical Imaging.

[12]  Sotirios A. Tsaftaris,et al.  TPSDicyc: Improved Deformation Invariant Cross-domain Medical Image Synthesis , 2019, MLMIR@MICCAI.

[13]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  J. Duryea,et al.  Magnetic resonance image segmentation using semi-automated software for quantification of knee articular cartilage—initial evaluation of a technique for paired scans , 2009, Skeletal Radiology.

[15]  Thomas M. Link,et al.  Isotropic 3D fast spin-echo imaging versus standard 2D imaging at 3.0 T of the knee—image quality and diagnostic performance , 2009, European Radiology.

[16]  H. Greenspan,et al.  CT Image-based Decision Support System for Categorization of Liver Metastases Into Primary Cancer Sites: Initial Results. , 2017, Academic radiology.

[17]  Harsh Agarwal,et al.  Automatic knee cartilage segmentation using fully volumetric convolutional neural networks for evaluation of osteoarthritis , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[18]  F Eckstein,et al.  Sensitivity to change of cartilage morphometry using coronal FLASH, sagittal DESS, and coronal MPR DESS protocols--comparative data from the Osteoarthritis Initiative (OAI). , 2010, Osteoarthritis and cartilage.

[19]  Max A. Viergever,et al.  Generative Adversarial Networks for Noise Reduction in Low-Dose CT , 2017, IEEE Transactions on Medical Imaging.

[20]  Snehashis Roy,et al.  Random forest regression for magnetic resonance image synthesis , 2017, Medical Image Anal..

[21]  Felix Eckstein,et al.  Imaging research results from the Osteoarthritis Initiative (OAI): a review and lessons learned 10 years after start of enrolment , 2014, Annals of the rheumatic diseases.

[22]  Dinggang Shen,et al.  Reconstruction of 7T-Like Images From 3T MRI , 2016, IEEE Transactions on Medical Imaging.

[23]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[24]  Sotirios A. Tsaftaris,et al.  Multimodal MR Synthesis via Modality-Invariant Latent Representation , 2018, IEEE Transactions on Medical Imaging.

[25]  Erika Schneider,et al.  The osteoarthritis initiative: report on the design rationale for the magnetic resonance imaging protocol for the knee. , 2008, Osteoarthritis and cartilage.

[26]  Sotirios A. Tsaftaris,et al.  Whole Image Synthesis Using a Deep Encoder-Decoder Network , 2016, SASHIMI@MICCAI.

[27]  Seungbum Koo,et al.  Recent advances in MRI of articular cartilage. , 2009, AJR. American journal of roentgenology.

[28]  Shaohua Kevin Zhou,et al.  Cross-Domain Synthesis of Medical Images Using Efficient Location-Sensitive Deep Network , 2015, MICCAI.

[29]  Christian Langkammer,et al.  DeepQSM - using deep learning to solve the dipole inversion for quantitative susceptibility mapping , 2019, NeuroImage.

[30]  Jeff Wood,et al.  Super‐resolution musculoskeletal MRI using deep learning , 2018, Magnetic resonance in medicine.

[31]  Mads Nielsen,et al.  Automatic segmentation of high- and low-field knee MRIs using knee image quantification with data from the osteoarthritis initiative , 2015, Journal of medical imaging.