Predicting Osteoarthritis Progression in Radiographs via Unsupervised Representation Learning

Osteoarthritis (OA) is the most common joint disorder affecting substantial proportions of the global population, primarily the elderly. Despite its individual and socioeconomic burden, the onset and progression of OA can still not be reliably predicted. Aiming to fill this diagnostic gap, we introduce an unsupervised learning scheme based on generative models to predict the future development of OA based on knee joint radiographs. Using longitudinal data from osteoarthritis studies, we explore the latent temporal trajectory to predict a patient’s future radiographs up to the eight-year follow-up visit. Our model predicts the risk of progression towards OA and surpasses its supervised counterpart whose input was provided by seven experienced radiologists. With the support of the model, sensitivity, specificity, positive predictive value, and negative predictive value increased significantly from 42.1% to 51.6%, from 72.3% to 88.6%, from 28.4% to 57.6%, and from 83.9% to 88.4%, respectively, while without such support, radiologists performed only slightly better than random guessing. Our predictive model improves predictions on OA onset and progression, despite requiring no human annotation in the training phase.

[1]  J. Tuomilehto,et al.  Functional Foods and Lifestyle Approaches for Diabetes Prevention and Management , 2017, Nutrients.

[2]  Ke Yan,et al.  Data augmentation using generative adversarial networks (CycleGAN) to improve generalizability in CT segmentation tasks , 2019, Scientific Reports.

[3]  D. Hassabis,et al.  Predicting conversion to wet age-related macular degeneration using deep learning , 2020, Nature Medicine.

[4]  M. Hochberg,et al.  Osteoarthritis I: epidemiology. , 1984, Maryland state medical journal.

[5]  Markus Zimmermann,et al.  Advancing diagnostic performance and clinical usability of neural networks via adversarial training and dual batch normalization , 2020, Nature Communications.

[6]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[7]  Lin Yang,et al.  Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  Sterling C. Johnson,et al.  Predicting Alzheimer’s disease progression using multi-modal deep learning approach , 2019, Scientific Reports.

[9]  Simo Saarakkala,et al.  KNEEL: Knee Anatomical Landmark Localization Using Hourglass Networks , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[10]  C. Cooper,et al.  UK clinical guideline for the prevention and treatment of osteoporosis , 2017, Archives of Osteoporosis.

[11]  Subarna Tripathi,et al.  Precise Recovery of Latent Vectors from Generative Adversarial Networks , 2017, ICLR.

[12]  Neil A Segal,et al.  The Multicenter Osteoarthritis Study: Opportunities for Rehabilitation Research , 2013, PM & R : the journal of injury, function, and rehabilitation.

[13]  Sepp Hochreiter,et al.  GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium , 2017, NIPS.

[14]  Jaakko Lehtinen,et al.  Analyzing and Improving the Image Quality of StyleGAN , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[16]  Fabian Isensee,et al.  Continuous-Time Deep Glioma Growth Models , 2021, MICCAI.

[17]  Sotirios A. Tsaftaris,et al.  DiCyc: GAN-based deformation invariant cross-domain information fusion for medical image synthesis , 2021, Inf. Fusion.

[18]  Simo Saarakkala,et al.  Multimodal Machine Learning-based Knee Osteoarthritis Progression Prediction from Plain Radiographs and Clinical Data , 2019, Scientific Reports.

[19]  David S. Melnick,et al.  International evaluation of an AI system for breast cancer screening , 2020, Nature.

[20]  Tobias Gauer,et al.  Radiomics of Brain MRI: Utility in Prediction of Metastatic Tumor Type. , 2019, Radiology.

[21]  P. Mildenberger,et al.  Introduction to the DICOM standard , 2002, European Radiology.

[22]  G. Marshall,et al.  Primary and Secondary Prevention Trials in Alzheimer Disease: Looking Back, Moving Forward. , 2016, Current Alzheimer research.

[23]  Klaus H. Maier-Hein,et al.  Deep Probabilistic Modeling of Glioma Growth , 2019, MICCAI.

[24]  Alexei A. Efros,et al.  Colorful Image Colorization , 2016, ECCV.

[25]  Olivier Bachem,et al.  Assessing Generative Models via Precision and Recall , 2018, NeurIPS.

[26]  Volkmar Schulz,et al.  Breaking medical data sharing boundaries by using synthesized radiographs , 2020, Science Advances.

[27]  Stanley Durrleman,et al.  Riemannian Geometry Learning for Disease Progression Modelling , 2019, IPMI.

[28]  A. Gelber,et al.  Osteoarthritis , 2020, Annals of Internal Medicine.

[29]  E. Pierson,et al.  An algorithmic approach to reducing unexplained pain disparities in underserved populations , 2021, Nature Medicine.

[30]  P. Conaghan Osteoarthritis in 2012: Parallel evolution of OA phenotypes and therapies , 2013, Nature Reviews Rheumatology.

[31]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[32]  Jeff Donahue,et al.  Large Scale Adversarial Representation Learning , 2019, NeurIPS.

[33]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[34]  E. Roos,et al.  Impact of type of meniscal tear on radiographic and symptomatic knee osteoarthritis: a sixteen-year followup of meniscectomy with matched controls. , 2003, Arthritis and rheumatism.

[35]  Steve Kench,et al.  Generating three-dimensional structures from a two-dimensional slice with generative adversarial network-based dimensionality expansion , 2021, Nature Machine Intelligence.

[36]  F. Berenbaum,et al.  Osteoarthritis: an update with relevance for clinical practice , 2011, The Lancet.

[37]  J. Lazarus,et al.  Burden of liver disease in Europe: Epidemiology and analysis of risk factors to identify prevention policies. , 2018, Journal of hepatology.

[38]  Lars Lau Raket,et al.  Statistical Disease Progression Modeling in Alzheimer Disease , 2020, Frontiers in Big Data.

[39]  Yang Lei,et al.  MRI-only based synthetic CT generation using dense cycle consistent generative adversarial networks. , 2019, Medical physics.

[40]  G. Lip,et al.  European Heart Rhythm Association (EHRA)/European Association of Cardiovascular Prevention and Rehabilitation (EACPR) position paper on how to prevent atrial fibrillation endorsed by the Heart Rhythm Society (HRS) and Asia Pacific Heart Rhythm Society (APHRS) , 2017, Europace : European pacing, arrhythmias, and cardiac electrophysiology : journal of the working groups on cardiac pacing, arrhythmias, and cardiac cellular electrophysiology of the European Society of Cardiology.

[41]  Felix Eckstein,et al.  Recent advances in osteoarthritis imaging—the Osteoarthritis Initiative , 2012, Nature Reviews Rheumatology.

[42]  René Rizzoli,et al.  Health economics in the field of osteoarthritis: an expert's consensus paper from the European Society for Clinical and Economic Aspects of Osteoporosis and Osteoarthritis (ESCEO). , 2013, Seminars in arthritis and rheumatism.

[43]  Martin Krzywinski,et al.  The curse(s) of dimensionality , 2018, Nature Methods.

[44]  Trevor Darrell,et al.  Adversarial Feature Learning , 2016, ICLR.

[45]  G. Corrado,et al.  End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography , 2019, Nature Medicine.