The Alzheimer's Disease Prediction Of Longitudinal Evolution (TADPOLE) Challenge: Results after 1 Year Follow-up

We present the findings of "The Alzheimer's Disease Prediction Of Longitudinal Evolution" (TADPOLE) Challenge, which compared the performance of 92 algorithms from 33 international teams at predicting the future trajectory of 219 individuals at risk of Alzheimer's disease. Challenge participants were required to make a prediction, for each month of a 5-year future time period, of three key outcomes: clinical diagnosis, Alzheimer's Disease Assessment Scale Cognitive Subdomain (ADAS-Cog13), and total volume of the ventricles. No single submission was best at predicting all three outcomes. For clinical diagnosis and ventricle volume prediction, the best algorithms strongly outperform simple baselines in predictive ability. However, for ADAS-Cog13 no single submitted prediction method was significantly better than random guessing. Two ensemble methods based on taking the mean and median over all predictions, obtained top scores on almost all tasks. Better than average performance at diagnosis prediction was generally associated with the additional inclusion of features from cerebrospinal fluid (CSF) samples and diffusion tensor imaging (DTI). On the other hand, better performance at ventricle volume prediction was associated with inclusion of summary statistics, such as patient-specific biomarker trends. The submission system remains open via the website this https URL, while code for submissions is being collated by TADPOLE SHARE: this https URL. Our work suggests that current prediction algorithms are accurate for biomarkers related to clinical diagnosis and ventricle volume, opening up the possibility of cohort refinement in clinical trials for Alzheimer's disease.

[1]  M. Jorge Cardoso,et al.  Robust parametric modeling of Alzheimer’s disease progression , 2019, NeuroImage.

[2]  Philippe Robert,et al.  Monotonic Gaussian Process for spatio-temporal disease progression modeling in brain imaging data , 2019, NeuroImage.

[3]  L. Rakêt,et al.  Disease progression modeling in Alzheimer’s disease: insights from the shape of cognitive decline , 2019 .

[4]  Razvan V. Marinescu,et al.  Differences in topological progression profile among neurodegenerative diseases from imaging data , 2019, eLife.

[5]  Kun Ho Lee,et al.  3D-Deep Learning Based Automatic Diagnosis of Alzheimer’s Disease with Joint MMSE Prediction Using Resting-State fMRI , 2019, Neuroinformatics.

[6]  Andrew J. Saykin,et al.  Deep Learning in Alzheimer's Disease: Diagnostic Classification and Prognostic Prediction Using Neuroimaging Data , 2019, Front. Aging Neurosci..

[7]  Manhua Liu,et al.  RNN-based longitudinal analysis for diagnosis of Alzheimer's disease , 2019, Comput. Medical Imaging Graph..

[8]  Keith A. Johnson,et al.  Predicting Alzheimer’s disease progression using multi-modal deep learning approach , 2019, Scientific Reports.

[9]  Daniel C. Alexander,et al.  DIVE: A spatiotemporal progression model of brain pathology in neurodegenerative disorders , 2019, NeuroImage.

[10]  Wiro J. Niessen,et al.  Disease progression timeline estimation for Alzheimer's disease using discriminative event based modeling , 2018, NeuroImage.

[11]  Maurizio Filippone,et al.  Probabilistic disease progression modeling to characterize diagnostic uncertainty: Application to staging and prediction in Alzheimer's disease , 2017, NeuroImage.

[12]  Di Guo,et al.  Convolutional Neural Networks-Based MRI Image Analysis for the Alzheimer’s Disease Prediction From Mild Cognitive Impairment , 2018, Front. Neurosci..

[13]  Aaron Carass,et al.  Why rankings of biomedical image analysis competitions should be interpreted with care , 2018, Nature Communications.

[14]  Nick C Fox,et al.  TADPOLE Challenge: Prediction of Longitudinal Evolution in Alzheimer's Disease. , 2018, 1805.03909.

[15]  Stanley Durrleman,et al.  Learning Distributions of Shape Trajectories from Longitudinal Datasets: A Hierarchical Model on a Manifold of Diffeomorphisms , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[16]  Daniel C Alexander,et al.  Data-driven models of dominantly-inherited Alzheimer’s disease progression , 2018, bioRxiv.

[17]  J. Ramírez,et al.  Machine-learning neuroimaging challenge for automated diagnosis of mild cognitive impairment: Lessons learnt , 2018, Journal of Neuroscience Methods.

[18]  S. Allassonnière,et al.  Simulating Alzheimer's disease progression with personalised digital brain models , 2018 .

[19]  Nick C Fox,et al.  Uncovering the heterogeneity and temporal complexity of neurodegenerative diseases with Subtype and Stage Inference , 2017, bioRxiv.

[20]  Stanley Durrleman,et al.  A Bayesian Mixed-Effects Model to Learn Trajectories of Changes from Repeated Manifold-Valued Observations , 2017, J. Mach. Learn. Res..

[21]  Peter F. Neher,et al.  The challenge of mapping the human connectome based on diffusion tractography , 2017, Nature Communications.

[22]  Antonio Cerasa,et al.  Random Forest Algorithm for the Classification of Neuroimaging Data in Alzheimer's Disease: A Systematic Review , 2017, Front. Aging Neurosci..

[23]  Dev Mehta,et al.  Why do trials for Alzheimer’s disease drugs keep failing? A discontinued drug perspective for 2010-2015 , 2017, Expert opinion on investigational drugs.

[24]  J. Morris,et al.  Recent publications from the Alzheimer's Disease Neuroimaging Initiative: Reviewing progress toward improved AD clinical trials , 2017, Alzheimer's & Dementia.

[25]  Chunxiang Jiang,et al.  Prediction and classification of Alzheimer disease based on quantification of MRI deformation , 2017, PloS one.

[26]  Joseph H. Grochowalski,et al.  Examining the reliability of ADAS-Cog change scores , 2016, Neuropsychology, development, and cognition. Section B, Aging, neuropsychology and cognition.

[27]  Gang Chen,et al.  Staging Alzheimer’s Disease Risk by Sequencing Brain Function and Structure, Cerebrospinal Fluid, and Cognition Biomarkers , 2016, Journal of Alzheimer's disease : JAD.

[28]  Norbert Schuff,et al.  Early role of vascular dysregulation on late-onset Alzheimer's disease based on multifactorial data-driven analysis , 2016, Nature Communications.

[29]  Michael W. Weiner,et al.  Crowdsourced estimation of cognitive decline and resilience in Alzheimer's disease , 2016, Alzheimer's & Dementia.

[30]  Jerry L. Prince,et al.  A multivariate nonlinear mixed effects model for longitudinal image analysis: Application to amyloid imaging , 2016, NeuroImage.

[31]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[32]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.

[33]  Vladimir Fonov,et al.  Standardized evaluation of algorithms for computer-aided diagnosis of dementia based on structural MRI: The CADDementia challenge , 2015, NeuroImage.

[34]  Heikki Huttunen,et al.  Machine learning framework for early MRI-based Alzheimer's conversion prediction in MCI subjects , 2015, NeuroImage.

[35]  Hélène Jacqmin-Gadda,et al.  Estimating long-term multivariate progression from short-term data , 2014, Alzheimer's & Dementia.

[36]  P. Mecocci,et al.  Random Forest ensembles for detection and prediction of Alzheimer's disease with a good between-cohort robustness , 2014, NeuroImage: Clinical.

[37]  Xiang Wang,et al.  Unsupervised learning of disease progression models , 2014, KDD.

[38]  Nick C Fox,et al.  A data-driven model of biomarker changes in sporadic Alzheimer's disease , 2014, Alzheimer's & Dementia.

[39]  Paul M. Thompson,et al.  Tractography density and network measures in Alzheimer'S disease , 2013, 2013 IEEE 10th International Symposium on Biomedical Imaging.

[40]  C. Rowe,et al.  Amyloid β deposition, neurodegeneration, and cognitive decline in sporadic Alzheimer's disease: a prospective cohort study , 2013, The Lancet Neurology.

[41]  Jerry L. Prince,et al.  A computational neurodegenerative disease progression score: Method and results with the Alzheimer's disease neuroimaging initiative cohort , 2012, NeuroImage.

[42]  Bruce Fischl,et al.  Within-subject template estimation for unbiased longitudinal image analysis , 2012, NeuroImage.

[43]  Sébastien Ourselin,et al.  An event-based model for disease progression and its application in familial Alzheimer's disease and Huntington's disease , 2012, NeuroImage.

[44]  W. Kukull,et al.  Accuracy of the Clinical Diagnosis of Alzheimer Disease at National Institute on Aging Alzheimer Disease Centers, 2005–2010 , 2012, Journal of neuropathology and experimental neurology.

[45]  M. Weiner,et al.  A Network Diffusion Model of Disease Progression in Dementia , 2012, Neuron.

[46]  Efstathios D. Gennatas,et al.  Predicting Regional Neurodegeneration from the Healthy Brain Functional Connectome , 2012, Neuron.

[47]  Peter Bühlmann,et al.  MissForest - non-parametric missing value imputation for mixed-type data , 2011, Bioinform..

[48]  M. Weiner,et al.  The dynamics of cortical and hippocampal atrophy in Alzheimer disease. , 2011, Archives of neurology.

[49]  Brian B. Avants,et al.  Evaluation of Registration Methods on Thoracic CT: The EMPIRE10 Challenge , 2011, IEEE Transactions on Medical Imaging.

[50]  Karsten Mueller,et al.  The General Linear Model , 2015 .

[51]  Michael Weiner,et al.  Docosahexaenoic acid supplementation and cognitive decline in Alzheimer disease: a randomized trial. , 2010, JAMA.

[52]  Roderick J A Little,et al.  A Review of Hot Deck Imputation for Survey Non‐response , 2010, International statistical review = Revue internationale de statistique.

[53]  Robert Tibshirani,et al.  Spectral Regularization Algorithms for Learning Large Incomplete Matrices , 2010, J. Mach. Learn. Res..

[54]  C. Jack,et al.  Hypothetical model of dynamic biomarkers of the Alzheimer's pathological cascade , 2010, The Lancet Neurology.

[55]  Liana G. Apostolova,et al.  Comparison of AdaBoost and Support Vector Machines for Detecting Alzheimer's Disease Through Automated Hippocampal Segmentation , 2010, IEEE Transactions on Medical Imaging.

[56]  C. Rowe,et al.  The Australian Imaging, Biomarkers and Lifestyle (AIBL) study of aging: methodology and baseline characteristics of 1112 individuals recruited for a longitudinal study of Alzheimer's disease , 2009, International Psychogeriatrics.

[57]  Arthur W. Toga,et al.  Atlas-based whole brain white matter analysis using large deformation diffeomorphic metric mapping: Application to normal elderly and Alzheimer's disease participants , 2009, NeuroImage.

[58]  Juan Manuel Górriz,et al.  Alzheimer's Diagnosis Using Eigenbrains and Support Vector Machines , 2009, IWANN.

[59]  D Salas-Gonzalez,et al.  Computer-aided diagnosis of Alzheimer's disease using support vector machines and classification trees , 2010, Physics in medicine and biology.

[60]  R. Bartha,et al.  Ventricular enlargement as a possible measure of Alzheimer's disease progression validated using the Alzheimer's disease neuroimaging initiative database. , 2008, Brain : a journal of neurology.

[61]  Nick C Fox,et al.  Automatic classification of MR scans in Alzheimer's disease. , 2008, Brain : a journal of neurology.

[62]  M. Pencina,et al.  Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond , 2008, Statistics in medicine.

[63]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[64]  Johan H. C. Reiber,et al.  Shape differences of the brain ventricles in Alzheimer's disease , 2006, NeuroImage.

[65]  Anders M. Dale,et al.  Reliability of MRI-derived measurements of human cerebral cortical thickness: The effects of field strength, scanner upgrade and manufacturer , 2006, NeuroImage.

[66]  H. Peitgen,et al.  Temporal horn index and volume of medial temporal lobe atrophy using a new semiautomated method for rapid and precise assessment. , 2006, AJNR. American journal of neuroradiology.

[67]  J. Quinn,et al.  Changes in premorbid brain volume predict Alzheimer’s disease pathology , 2003, Neurology.

[68]  Nick C Fox,et al.  Mapping the evolution of regional atrophy in Alzheimer's disease: Unbiased analysis of fluid-registered serial MRI , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[69]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[70]  R. Tibshirani,et al.  An Introduction to the Bootstrap , 1995 .

[71]  E. DeLong,et al.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. , 1988, Biometrics.

[72]  J. Kalbfleisch,et al.  The Analysis of Panel Data under a Markov Assumption , 1985 .