Improving Robustness of Deep Learning Based Knee MRI Segmentation: Mixup and Adversarial Domain Adaptation

Degeneration of articular cartilage (AC) is actively studied in knee osteoarthritis (OA) research via magnetic resonance imaging (MRI). Segmentation of AC tissues from MRI data is an essential step in quantification of their damage. Deep learning (DL) based methods have shown potential in this realm and are the current state-of-the-art, however, their robustness to heterogeneity of MRI acquisition settings remains an open problem. In this study, we investigated two modern regularization techniques – mixup and adversarial unsupervised domain adaptation (UDA) – to improve the robustness of DL-based knee cartilage segmentation to new MRI acquisition settings. Our validation setup included two datasets produced by different MRI scanners and using distinct data acquisition protocols. We assessed the robustness of automatic segmentation by comparing mixup and UDA approaches to a strong baseline method at different OA severity stages and, additionally, in relation to anatomical locations. Our results showed that for moderate changes in knee MRI data acquisition settings both approaches may provide notable improvements in the robustness, which are consistent for all stages of the disease and affect the clinically important areas of the knee joint. However, mixup may be considered as a recommended approach, since it is more computationally efficient and does not require additional data from the target acquisition setup.

[1]  Stefan Zachow,et al.  Automated segmentation of knee bone and cartilage combining statistical shape knowledge and convolutional neural networks: Data from the Osteoarthritis Initiative , 2019, Medical Image Anal..

[2]  Hao Chen,et al.  Unsupervised Cross-Modality Domain Adaptation of ConvNets for Biomedical Image Segmentations with Adversarial Loss , 2018, IJCAI.

[3]  Hongyi Zhang,et al.  mixup: Beyond Empirical Risk Minimization , 2017, ICLR.

[4]  Richard Kijowski,et al.  Deep convolutional neural network and 3D deformable approach for tissue segmentation in musculoskeletal magnetic resonance imaging , 2018, Magnetic resonance in medicine.

[5]  Yoichi Yaguchi,et al.  MixFeat: Mix Feature in Latent Space Learns Discriminative Space , 2018 .

[6]  Andre Esteva,et al.  A guide to deep learning in healthcare , 2019, Nature Medicine.

[7]  M. Jorge Cardoso,et al.  Improving Data Augmentation for Medical Image Segmentation , 2018 .

[8]  Simo Saarakkala,et al.  Deep-Learning for Tidemark Segmentation in Human Osteochondral Tissues Imaged with Micro-computed Tomography , 2019, ACIVS.

[9]  Janet L. Ronsky,et al.  Establishing outcome measures in early knee osteoarthritis , 2019, Nature Reviews Rheumatology.

[10]  Louise Rainford,et al.  3T MRI of the knee with optimised isotropic 3D sequences: Accurate delineation of intra-articular pathology without prolonged acquisition times , 2017, European Radiology.

[11]  Hao Chen,et al.  Semantic-Aware Generative Adversarial Nets for Unsupervised Domain Adaptation in Chest X-ray Segmentation , 2018, MLMI@MICCAI.

[12]  Stefan Zachow,et al.  Accurate Automated Volumetry of Cartilage of the Knee Using Convolutional Neural Networks: Data From the Osteoarthritis Initiative , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[13]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Erika Schneider,et al.  The osteoarthritis initiative: report on the design rationale for the magnetic resonance imaging protocol for the knee. , 2008, Osteoarthritis and cartilage.

[15]  Simo Saarakkala,et al.  Multimodal Machine Learning-based Knee Osteoarthritis Progression Prediction from Plain Radiographs and Clinical Data , 2019, Scientific Reports.

[16]  Marc Niethammer,et al.  DeepAtlas: Joint Semi-Supervised Learning of Image Registration and Segmentation , 2019, MICCAI.

[17]  Daniel Marcu,et al.  Domain Adaptation for Statistical Classifiers , 2006, J. Artif. Intell. Res..

[18]  Simo Saarakkala,et al.  Automatic Knee Osteoarthritis Diagnosis from Plain Radiographs: A Deep Learning-Based Approach , 2017, Scientific Reports.

[19]  J. Niinimäki,et al.  Comparison of Diagnostic Performance of Semi-Quantitative Knee Ultrasound and Knee Radiography with MRI: Oulu Knee Osteoarthritis Study , 2016, Scientific Reports.

[20]  Ioannis Mitliagkas,et al.  Manifold Mixup: Better Representations by Interpolating Hidden States , 2018, ICML.

[21]  V. Casula,et al.  Osteoarthritis year in review 2018: imaging. , 2019, Osteoarthritis and cartilage.

[22]  Daniel Cremers,et al.  Regularization for Deep Learning: A Taxonomy , 2017, ArXiv.

[23]  N. Subhas,et al.  3D MRI in Musculoskeletal Imaging: Current and Future Applications , 2018, Current Radiology Reports.

[24]  M. D. Ryzhkov,et al.  Knee Cartilage Segmentation Algorithms: a Critical Literature Review , 2015 .

[25]  Trevor Darrell,et al.  Deep Domain Confusion: Maximizing for Domain Invariance , 2014, CVPR 2014.

[26]  Ender Konukoglu,et al.  A Lifelong Learning Approach to Brain MR Segmentation Across Scanners and Protocols , 2018, MICCAI.

[27]  T Stammberger,et al.  Interobserver reproducibility of quantitative cartilage measurements: comparison of B-spline snakes and manual segmentation. , 1999, Magnetic resonance imaging.

[28]  Mads Nielsen,et al.  Automatic segmentation of high- and low-field knee MRIs using knee image quantification with data from the osteoarthritis initiative , 2015, Journal of medical imaging.

[29]  Marleen de Bruijne,et al.  Grey Matter Age Prediction as a Biomarker for Risk of Dementia: A Population-based Study , 2019, bioRxiv.

[30]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[31]  Hongyu Guo,et al.  MixUp as Locally Linear Out-Of-Manifold Regularization , 2018, AAAI.

[32]  Simo Saarakkala,et al.  Automatic Grading of Individual Knee Osteoarthritis Features in Plain Radiographs Using Deep Convolutional Neural Networks , 2019, Diagnostics.

[33]  S. Majumdar,et al.  Use of 2D U-Net Convolutional Neural Networks for Automated Cartilage and Meniscus Segmentation of Knee MR Imaging Data to Determine Relaxometry and Morphometry. , 2018, Radiology.

[34]  Ming-Hsuan Yang,et al.  Learning to Adapt Structured Output Space for Semantic Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[35]  Sumit Chopra,et al.  DLID: Deep Learning for Domain Adaptation by Interpolating between Domains , 2013 .

[36]  Eric K. Gibbons,et al.  Utility of deep learning super‐resolution in the context of osteoarthritis MRI biomarkers , 2020, Journal of magnetic resonance imaging : JMRI.

[37]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[38]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[39]  J. Kellgren,et al.  Radiological Assessment of Osteo-Arthrosis , 1957, Annals of the rheumatic diseases.

[40]  Ender Konukoglu,et al.  Semi-Supervised and Task-Driven Data Augmentation , 2019, IPMI.

[41]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[42]  Ali Mobasheri,et al.  Osteoarthritis phenotypes and novel therapeutic targets. , 2019, Biochemical pharmacology.

[43]  Ioannis Mitliagkas,et al.  Manifold Mixup: Encouraging Meaningful On-Manifold Interpolation as a Regularizer , 2018, ArXiv.

[44]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[45]  Fang Liu,et al.  SUSAN: segment unannotated image structure using adversarial network , 2018, Magnetic resonance in medicine.

[46]  S Zachow,et al.  Knee menisci segmentation using convolutional neural networks: data from the Osteoarthritis Initiative. , 2018, Osteoarthritis and cartilage.

[47]  Brian A. Hargreaves,et al.  Technical Considerations for Semantic Segmentation in MRI using Convolutional Neural Networks , 2019, ArXiv.

[48]  Sanjay N. Talbar,et al.  Knee Articular Cartilage Segmentation from MR Images , 2018, ACM Comput. Surv..