Optimum threshold estimation based on cost function in a multistate diagnostic setting

In the diagnostic area, the usual setting considers two populations: nondiseased and diseased. The use of the standard ROC analysis methodology is well established. Sometimes, however, diagnostic problems inherently include more than two classification states. For example, 'yes, uncertain, no' or 'low, normal, high'. Here we consider a three-normal distribution setting and derive estimators for the optimum thresholds between states based on a cost function. These estimators can be extended for clinical contexts with more than three states. This approach is well known for the two-state setting and its advantage lies in the fact that it accounts for the specific context's properties, such as disease prevalence and classification costs. Here we calculated the variance of the estimators by the use of parametric methods on nonlinear equations and we constructed confidence intervals accounting for possible uncertainty in the threshold estimation. We conducted a simulation study to assess the performance of these estimators and the confidence intervals. Comparisons with the naive threshold estimation method of joining the distributions two-by-two and applying standard ROC techniques proved that the latter method is not reliable for all parameter combinations and should be avoided.