Towards Trustworthy Predictions of Conversion from Mild Cognitive Impairment to Dementia: A Conformal Prediction Approach

Predicting progression from a stage of Mild Cognitive Impairment to Alzheimer’s disease is a major pursuit in current dementia research. As a result, many prognostic models have emerged with the goal of supporting clinical decisions. Despite the efforts, the lack of a reliable assessment of the uncertainty of each prediction has hampered its application in practise. It is paramount for clinicians to know how much they can rely upon the prediction made for a given patient, in order to adjust treatments to the patient based on that information. In this exploratory study, we evaluated the Conformal Prediction approach on the task of making predictions with precise levels of confidence. Conformal prediction showed promising results. Using high confidence levels have the drawback of leaving a large number of MCI patients without prognostic (the classifier is not confident enough to give a single class). When using forced predictions, conformal predictors achieved classification performances as good as standard classifiers, with the advantage of complementing each prediction with a confidence value.