Extensions of ROC Analysis to multi-class domains

Receiver-operating characteristic (ROC) analysis has proven to be a powerful method for dealing with misclassification costs and skewed class distributions (Provost & Fawcett, 1998). In the typical representation, an ROC analysis evaluates false accept versus false reject rates for a set of candidate binary classifiers under all possible (linear) cost and prior class distribution assumptions. The set of potentially optimal classifiers forms a convex hull (the ROCCH) in the 2-space of error rates, and a particular cost matrix and class distribution vector yields a linear surface tangent to the surface of the ROC hull. The set of points at the tangent of the ROCCH and the cost-surface constitutes the set of optimal classifiers for that particular cost assumption. Classifiers corresponding to points not on the hull are never optimal under any conditions and can be discarded without further consideration. Classifier-construction algorithms yielding a set of classifiers all of whom fall strictly below the ROCCH are said to be dominated by one or more other algorithms and can also be discarded from consideration.