Testing non‐inferiority (and equivalence) between two diagnostic procedures in paired‐sample ordinal data

Before adopting a new diagnostic procedure, which is more convenient and less expensive than the standard existing procedure, it is essentially important to assess whether the diagnostic accuracy of the new procedure is non‐inferior (or equivalent) to that of the standard procedure. In this paper, we consider the situation where test responses are on an ordinal scale with more than two categories. We give two definitions of non‐inferiority, one in terms of the probability of correctly identifying the case for a randomly selected pair of a case and a non‐case over all possible cut‐off points, and the other in terms of both the sensitivity and specificity directly. On the basis of large sample theory, we develop two simple test procedures for detecting non‐inferiority. We further conduct Monte Carlo simulation to evaluate the finite sample performance of these test procedures. We note that the two asymptotic test procedures proposed here can actually perform reasonably well in a variety of situations even when the numbers of studied subjects from the diseased and non‐diseased populations are not large. To illustrate the use of the proposed test procedures, we include an example of determining whether the diagnostic accuracy of using a digitized film is non‐inferior to that of using a plain film for screening breast cancer. Finally, we note that the extension of these results to accommodate the case of detecting (two‐sided) equivalence is simply straightforward. Copyright © 2004 John Wiley & Sons, Ltd.