Repeatability profiles towards consistent sensitivity and specificity levels for machine learning on breast DCE-MRI

We evaluated a radiomics/machine learning method for dynamic contrast-enhanced magnetic resonance (DCE-MR) images of breast lesions and the impact of case-based classification repeatability on sensitivity and specificity. DCE-MR images of 1,169 unique breast lesions (267 benign, 902 malignant) were retrospectively collected under HIPAA/IRB. Lesions were automatically segmented using a fuzzy c-means method; thirty-eight radiomic features were extracted. Three classification tasks were investigated: (i) benign vs. malignant, (ii) pure ductal carcinoma in situ (DCIS) vs. DCIS with invasive ductal carcinoma (IDC), and (iii) luminal A or luminal B cancers vs. other molecular subtypes. Case-based repeatability of classifier output was constructed using 0.632+ bootstrap sampling (1000 iterations) with classification by support vector machine (SVM). Repeatability profiles were constructed for each task using the 95% confidence interval widths of the classifier output for cases in the test folds over all bootstrap iterations. The relationships between classifier output repeatability and variability in sensitivity and specificity over the bootstrap test folds were investigated. Most cases fell within the highest repeatability of classifier output over all three classification tasks. Sensitivity and specificity demonstrated more variability in the test folds than in the training folds at corresponding thresholds for the classifier output. Higher repeatability of classifier output was associated with lower variability in sensitivity and specificity in tasks (i) and (ii) but not in task (iii). Case-based repeatability profiles may be important for characterizing impact of using radiomics with desired sensitivity and specificity.