Improving Robustness of Deep Learning Based Knee MRI Segmentation: Mixup and Adversarial Domain Adaptation

Degeneration of articular cartilage (AC) is actively studied in knee osteoarthritis (OA) research via magnetic resonance imaging (MRI). Segmentation of AC tissues from MRI data is an essential step in quantification of their damage. Deep learning (DL) based methods have shown potential in this realm and are the current state-of-the-art, however, their robustness to heterogeneity of MRI acquisition settings remains an open problem. In this study, we investigated two modern regularization techniques – mixup and adversarial unsupervised domain adaptation (UDA) – to improve the robustness of DL-based knee cartilage segmentation to new MRI acquisition settings. Our validation setup included two datasets produced by different MRI scanners and using distinct data acquisition protocols. We assessed the robustness of automatic segmentation by comparing mixup and UDA approaches to a strong baseline method at different OA severity stages and, additionally, in relation to anatomical locations. Our results showed that for moderate changes in knee MRI data acquisition settings both approaches may provide notable improvements in the robustness, which are consistent for all stages of the disease and affect the clinically important areas of the knee joint. However, mixup may be considered as a recommended approach, since it is more computationally efficient and does not require additional data from the target acquisition setup.

[1]  Eric K. Gibbons,et al.  Utility of deep learning super‐resolution in the context of osteoarthritis MRI biomarkers , 2020, Journal of magnetic resonance imaging : JMRI.

[2]  Simo Saarakkala,et al.  Automatic Grading of Individual Knee Osteoarthritis Features in Plain Radiographs Using Deep Convolutional Neural Networks , 2019, Diagnostics.

[3]  Simo Saarakkala,et al.  Deep-Learning for Tidemark Segmentation in Human Osteochondral Tissues Imaged with Micro-computed Tomography , 2019, ACIVS.

[4]  Ali Mobasheri,et al.  Osteoarthritis phenotypes and novel therapeutic targets. , 2019, Biochemical pharmacology.

[5]  Janet L. Ronsky,et al.  Establishing outcome measures in early knee osteoarthritis , 2019, Nature Reviews Rheumatology.

[6]  Marc Niethammer,et al.  DeepAtlas: Joint Semi-Supervised Learning of Image Registration and Segmentation , 2019, MICCAI.

[7]  Simo Saarakkala,et al.  Multimodal Machine Learning-based Knee Osteoarthritis Progression Prediction from Plain Radiographs and Clinical Data , 2019, Scientific Reports.

[8]  Ender Konukoglu,et al.  Semi-Supervised and Task-Driven Data Augmentation , 2019, IPMI.

[9]  Brian A. Hargreaves,et al.  Technical Considerations for Semantic Segmentation in MRI using Convolutional Neural Networks , 2019, ArXiv.

[10]  Stefan Zachow,et al.  Automated segmentation of knee bone and cartilage combining statistical shape knowledge and convolutional neural networks: Data from the Osteoarthritis Initiative , 2019, Medical Image Anal..

[11]  Stefan Zachow,et al.  Accurate Automated Volumetry of Cartilage of the Knee Using Convolutional Neural Networks: Data From the Osteoarthritis Initiative , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[12]  Marleen de Bruijne,et al.  Grey Matter Age Prediction as a Biomarker for Risk of Dementia: A Population-based Study , 2019, bioRxiv.

[13]  Andre Esteva,et al.  A guide to deep learning in healthcare , 2019, Nature Medicine.

[14]  Fang Liu,et al.  SUSAN: segment unannotated image structure using adversarial network , 2018, Magnetic resonance in medicine.

[15]  Sanjay N. Talbar,et al.  Knee Articular Cartilage Segmentation from MR Images , 2018, ACM Comput. Surv..

[16]  Yoichi Yaguchi,et al.  MixFeat: Mix Feature in Latent Space Learns Discriminative Space , 2018 .

[17]  Hongyu Guo,et al.  MixUp as Locally Linear Out-Of-Manifold Regularization , 2018, AAAI.

[18]  Ioannis Mitliagkas,et al.  Manifold Mixup: Better Representations by Interpolating Hidden States , 2018, ICML.

[19]  Ioannis Mitliagkas,et al.  Manifold Mixup: Encouraging Meaningful On-Manifold Interpolation as a Regularizer , 2018, ArXiv.

[20]  Hao Chen,et al.  Semantic-Aware Generative Adversarial Nets for Unsupervised Domain Adaptation in Chest X-ray Segmentation , 2018, MLMI@MICCAI.

[21]  N. Subhas,et al.  3D MRI in Musculoskeletal Imaging: Current and Future Applications , 2018, Current Radiology Reports.

[22]  Ender Konukoglu,et al.  A Lifelong Learning Approach to Brain MR Segmentation Across Scanners and Protocols , 2018, MICCAI.

[23]  S Zachow,et al.  Knee menisci segmentation using convolutional neural networks: data from the Osteoarthritis Initiative. , 2018, Osteoarthritis and cartilage.

[24]  Hao Chen,et al.  Unsupervised Cross-Modality Domain Adaptation of ConvNets for Biomedical Image Segmentations with Adversarial Loss , 2018, IJCAI.

[25]  M. Jorge Cardoso,et al.  Improving Data Augmentation for Medical Image Segmentation , 2018 .

[26]  Miika T. Nieminen,et al.  Osteoarthritis year in review 2018: imaging. , 2019, Osteoarthritis and cartilage.

[27]  Richard Kijowski,et al.  Deep convolutional neural network and 3D deformable approach for tissue segmentation in musculoskeletal magnetic resonance imaging , 2018, Magnetic resonance in medicine.

[28]  S. Majumdar,et al.  Use of 2D U-Net Convolutional Neural Networks for Automated Cartilage and Meniscus Segmentation of Knee MR Imaging Data to Determine Relaxometry and Morphometry. , 2018, Radiology.

[29]  Ming-Hsuan Yang,et al.  Learning to Adapt Structured Output Space for Semantic Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[30]  Simo Saarakkala,et al.  Automatic Knee Osteoarthritis Diagnosis from Plain Radiographs: A Deep Learning-Based Approach , 2017, Scientific Reports.

[31]  Daniel Cremers,et al.  Regularization for Deep Learning: A Taxonomy , 2017, ArXiv.

[32]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[33]  Hongyi Zhang,et al.  mixup: Beyond Empirical Risk Minimization , 2017, ICLR.

[34]  Louise Rainford,et al.  3T MRI of the knee with optimised isotropic 3D sequences: Accurate delineation of intra-articular pathology without prolonged acquisition times , 2017, European Radiology.

[35]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[36]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  J. Niinimäki,et al.  Comparison of Diagnostic Performance of Semi-Quantitative Knee Ultrasound and Knee Radiography with MRI: Oulu Knee Osteoarthritis Study , 2016, Scientific Reports.

[38]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[39]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[40]  Mads Nielsen,et al.  Automatic segmentation of high- and low-field knee MRIs using knee image quantification with data from the osteoarthritis initiative , 2015, Journal of medical imaging.

[41]  M. D. Ryzhkov,et al.  Knee Cartilage Segmentation Algorithms: a Critical Literature Review , 2015 .

[42]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[43]  Trevor Darrell,et al.  Deep Domain Confusion: Maximizing for Domain Invariance , 2014, CVPR 2014.

[44]  Erika Schneider,et al.  The osteoarthritis initiative: report on the design rationale for the magnetic resonance imaging protocol for the knee. , 2008, Osteoarthritis and cartilage.

[45]  Daniel Marcu,et al.  Domain Adaptation for Statistical Classifiers , 2006, J. Artif. Intell. Res..

[46]  T Stammberger,et al.  Interobserver reproducibility of quantitative cartilage measurements: comparison of B-spline snakes and manual segmentation. , 1999, Magnetic resonance imaging.

[47]  J. Kellgren,et al.  Radiological Assessment of Osteo-Arthrosis , 1957, Annals of the rheumatic diseases.

[48]  Sumit Chopra,et al.  DLID: Deep Learning for Domain Adaptation by Interpolating between Domains , 2013 .