Assessment of knee pain from MRI scans using a convolutional Siamese network

ABSTRACT Objectives It remains difficult to characterize pain in knee joints with risk of osteoarthritis solely by radiographic findings. We sought to understand if advanced machine learning methods such as deep neural networks can be used to predict and identify the structural features that are associated with knee pain. Methods We constructed a convolutional Siamese network to associate MRI scans obtained on subjects from the Osteoarthritis Initiative (OAI) with frequent unilateral knee pain (n=1,529) comparing their knee with frequent pain to the contralateral knee without pain. The Siamese network architecture enabled pairwise learning of information from two-dimensional (2D) sagittal intermediate-weighted turbo spin echo slices obtained from similar locations on both knees. Class activation mapping (CAM) was utilized to create saliency maps, which highlighted the regions that were most associated with knee pain. The MRI scans and the CAMs of each subject were reviewed by a radiologist to identify the presence of abnormalities within the model-predicted regions of high association. Results Using 10-fold cross validation, our model achieved an area under curve (AUC) value of 0.808. When individuals whose knee WOMAC pain scores were not discordant were excluded, model performance increased to 0.853. The radiologist review revealed that about 86% of the cases that were predicted correctly had effusion-synovitis within the regions that were most associated with pain. Conclusions This study demonstrates a proof of principle that deep learning can be applied to assess knee pain from MRI scans.

[1]  Philip G Conaghan,et al.  Imaging the painful osteoarthritic knee joint: what have we learned? , 2009, Nature Clinical Practice Rheumatology.

[2]  Felix Eckstein,et al.  Magnetic resonance imaging-based cartilage loss in painful contralateral knees with and without radiographic joint space narrowing: Data from the Osteoarthritis Initiative. , 2009, Arthritis and rheumatism.

[3]  C. Hayes,et al.  Magnetic resonance-detected subchondral bone marrow and cartilage defect characteristics associated with pain and X-ray-defined knee osteoarthritis. , 2003, Osteoarthritis and cartilage.

[4]  C. McCulloch,et al.  Individual magnetic resonance imaging and radiographic features of knee osteoarthritis in subjects with unilateral knee pain: the health, aging, and body composition study. , 2012, Arthritis and rheumatism.

[5]  A. Guermazi,et al.  Structural correlates of pain in joints with osteoarthritis. , 2013, Osteoarthritis and cartilage.

[6]  J. Singer,et al.  Natural history of cartilage damage and osteoarthritis progression on magnetic resonance imaging in a population-based cohort with knee pain. , 2011, Osteoarthritis and cartilage.

[7]  Erika Schneider,et al.  The osteoarthritis initiative: report on the design rationale for the magnetic resonance imaging protocol for the knee. , 2008, Osteoarthritis and cartilage.

[8]  A. Guermazi,et al.  The association between meniscal and cruciate ligament damage and knee pain in community residents. , 2011, Osteoarthritis and cartilage.

[9]  F. Fawaz-Estrup.  The osteoarthritis initiative: an overview. , 2004, Medicine and health, Rhode Island.

[10]  A Guermazi,et al.  Effect of meniscal damage on the development of frequent knee pain, aching, or stiffness. , 2007, Arthritis and rheumatism.

[11]  David Felson,et al.  Association between radiographic features of knee osteoarthritis and pain: results from two cohort studies , 2009, BMJ : British Medical Journal.

[12]  Lipo Wang,et al.  Deep Learning Applications in Medical Image Analysis , 2018, IEEE Access.

[13]  S. Majumdar,et al.  Use of 2D U-Net Convolutional Neural Networks for Automated Cartilage and Meniscus Segmentation of Knee MR Imaging Data to Determine Relaxometry and Morphometry. , 2018, Radiology.

[14]  Yann LeCun,et al.  Learning a similarity metric discriminatively, with application to face verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[15]  Nima Tajbakhsh,et al.  Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? , 2016, IEEE Transactions on Medical Imaging.

[16]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[17]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[18]  Tim Cootes,et al.  Comparing image analysis approaches versus expert readers: the relation of knee radiograph features to knee pain , 2018, Annals of the Rheumatic Diseases.

[19]  G. Kitas,et al.  Neuropathic pain in osteoarthritis: a review of pathophysiological mechanisms and implications for treatment. , 2014, Seminars in arthritis and rheumatism.

[20]  L. Kazis,et al.  The Association of Bone Marrow Lesions with Pain in Knee Osteoarthritis , 2001, Annals of Internal Medicine.

[21]  T. Vos,et al.  The global burden of hip and knee osteoarthritis: estimates from the Global Burden of Disease 2010 study , 2014, Annals of the rheumatic diseases.

[22]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[23]  Yan Chai Hum,et al.  Train Convolutional Neural Networks Without Well-Segmented Ground Truth Images for Cartilage Localization: Data from the Osteoarthritis Initiatives , 2018 .

[24]  S. Goldring,et al.  Osteoarthritis: a disease of the joint as an organ. , 2012, Arthritis and rheumatism.

[25]  Priyamvada Singh,et al.  Association of Pathological Fibrosis With Renal Survival Using Deep Neural Networks , 2018, Kidney international reports.

[26]  D. Felson,et al.  Mechanisms of Osteoarthritis (OA) Pain , 2018, Current Osteoporosis Reports.

[27]  Jose Dolz,et al.  3D fully convolutional networks for subcortical segmentation in MRI: A large-scale study , 2016, NeuroImage.

[28]  H. Genant,et al.  Synovitis detected on magnetic resonance imaging and its relation to pain and cartilage loss in knee osteoarthritis , 2007, Annals of the rheumatic diseases.

[29]  Arno Klein,et al.  Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration , 2009, NeuroImage.

[30]  Ali Guermazi,et al.  Correlation of the development of knee pain with enlarging bone marrow lesions on magnetic resonance imaging. , 2007, Arthritis and rheumatism.

[31]  J. Singer,et al.  Associations between MRI features versus knee pain severity and progression: Data from the Vancouver Longitudinal Study of Early Knee Osteoarthritis , 2017, PloS one.

[32]  T. Huizinga,et al.  Do knee abnormalities visualised on MRI explain knee pain in knee osteoarthritis? A systematic review , 2010, Annals of the rheumatic diseases.

[33]  L. Kazis,et al.  Knee effusions, popliteal cysts, and synovial thickening: association with knee pain in osteoarthritis. , 2001, The Journal of rheumatology.

[34]  Simo Saarakkala,et al.  Automatic Knee Osteoarthritis Diagnosis from Plain Radiographs: A Deep Learning-Based Approach , 2017, Scientific Reports.

[35]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  D. Clauw,et al.  Pain and rheumatology: thinking outside the joint. , 2009, Arthritis and rheumatism.

[37]  R. Edwards,et al.  Discordance between pain and radiographic severity in knee osteoarthritis: findings from quantitative sensory testing of central sensitization. , 2013, Arthritis and rheumatism.

[38]  John Bedson,et al.  The discordance between clinical and radiographic knee osteoarthritis: A systematic search and summary of the literature , 2008, BMC musculoskeletal disorders.