Predicting Knee Osteoarthritis Progression from Structural MRI Using Deep Learning

Accurate prediction of knee osteoarthritis (KOA) progression from structural MRI has a potential to enhance disease understanding and support clinical trials. Prior art focused on manually designed imaging biomarkers, which may not fully exploit all disease-related information present in MRI scan. In contrast, our method learns relevant representations from raw data end-to-end using Deep Learning, and uses them for progression prediction. The method employs a 2D CNN to process the data slice-wise and aggregate the extracted features using a Transformer. Evaluated on a large cohort (n=4,866), the proposed method outperforms conventional 2D and 3D CNN-based models and achieves average precision of 0.58 ± 0.03 and ROC AUC of 0.78 ± 0.01. This paper sets a baseline on end-to-end KOA progression prediction from structural MRI. Our code is publicly available at https://github.com/MIPT-Oulu/OAProgressionMR.

[1]  Cordelia Schmid,et al.  ViViT: A Video Vision Transformer , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[2]  Simo Saarakkala,et al.  Multimodal Machine Learning-based Knee Osteoarthritis Progression Prediction from Plain Radiographs and Clinical Data , 2019, Scientific Reports.

[3]  Junlong Zhong,et al.  Global, regional prevalence, incidence and risk factors of knee osteoarthritis in population-based studies , 2020, EClinicalMedicine.

[4]  S. Majumdar,et al.  Deep learning for large scale MRI-based morphological phenotyping of osteoarthritis , 2021, Scientific Reports.

[5]  Ahmet Gunduz,et al.  Resource Efficient 3D Convolutional Neural Networks , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[6]  Gregory Chang,et al.  Total Knee Replacement prediction using Structural MRIs and 3D Convolutional Neural Networks , 2019 .

[7]  M. Haapea,et al.  Elevated adiabatic T1ρ and T2ρ in articular cartilage are associated with cartilage and bone lesions in early osteoarthritis: A preliminary study , 2017, Journal of magnetic resonance imaging : JMRI.

[8]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Tassilo Klein,et al.  Multimodal Self-Supervised Learning for Medical Image Analysis , 2019, IPMI.

[10]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  J. Kellgren,et al.  Radiological Assessment of Rheumatoid Arthritis * , 1957, Annals of the rheumatic diseases.

[12]  Georg Heigold,et al.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2021, ICLR.

[13]  Sharmila Majumdar,et al.  Deep Learning Predicts Total Knee Replacement from Magnetic Resonance Images , 2020, Scientific Reports.

[14]  Peng Cao,et al.  Learning osteoarthritis imaging biomarkers from bone surface spherical encoding , 2020, Magnetic resonance in medicine.

[15]  Takaya Saito,et al.  The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets , 2015, PloS one.

[16]  Neil Houlsby,et al.  Supervised Transfer Learning at Scale for Medical Imaging , 2021, ArXiv.

[17]  Yann LeCun,et al.  A Closer Look at Spatiotemporal Convolutions for Action Recognition , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Huiye Liu,et al.  TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation , 2021, MICCAI.

[19]  Akshay Pai,et al.  One Network to Segment Them All: A General, Lightweight System for Accurate 3D Medical Image Segmentation , 2019, MICCAI.

[20]  Shekoofeh Azizi,et al.  Big Self-Supervised Models Advance Medical Image Classification , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[21]  Chunhua Shen,et al.  CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation , 2021, MICCAI.

[22]  Taghi M. Khoshgoftaar,et al.  Survey on deep learning with class imbalance , 2019, J. Big Data.

[23]  Jon Kleinberg,et al.  Transfusion: Understanding Transfer Learning for Medical Imaging , 2019, NeurIPS.