ROC Graphs for Assessing the Ability of a Diagnostic Marker to Detect Three Disease Classes with an Umbrella Ordering

Receiver operating characteristic (ROC) curves and the area under these curves are commonly used to assess the ability of a continuous diagnostic marker (e.g., DNA methylation markers) to correctly classify subjects as having a particular disease or not (e.g., cancer). These approaches, however, are not applicable to settings where the gold standard yields more than two disease states or classes. ROC surfaces and the volume under the surfaces have been proposed for settings with more than two disease classes. These approaches, however, do not allow one to assess the ability of a marker to differentiate two disease classes from a third disease class without requiring a monotone order for the three disease classes under study. That is, existing approaches do not accommodate an umbrella ordering of disease classes. This article proposes the construction of an ROC graph that is applicable for an umbrella ordering. Furthermore, this article proposes that a summary measure for this umbrella ROC graph can be used to summarize the classification accuracy, and corresponding variance estimates can be obtained using U-statistics theory or bootstrap methods. The proposed methods are illustrated using data from a study assessing the ability of a DNA methylation marker to correctly classify lung specimens into three histologic classes: squamous cell carcinoma, large cell carcinoma, and nontumor lung.