On the repeated use of databases for testing incremental improvement of computer-aided detection schemes.

Imaging-based computer-aided detection (CAD) or computer-aided diagnosis (CADx) technology is continuing to improve and it appears inevitable that it will merge into the routine clinical practice of diagnostic medicine in general and radiology in particular. (For simplicity we will use CAD here to identify these technologies generically.) Today, CAD use is expanding rapidly in the clinical environment and the number of different applications is likely to increase in the near future. The more obvious examples in the radiologic domain are conventional and digital mammography, conventional and digital chest radiography, computed tomography of the lung, and optical and virtual colonoscopy. Over the last few years we have seen several CAD products approved or cleared for marketing by the US Food and Drug Administration. Most CAD systems use tools regarded as a sub-class of the broad field of statistical learning machines (SLM). Such machines require the use of many exemplar cases — the so-called “training dataset” — during the development or training phase when an attempt is made to optimize the learning and performance of a specified task based on this “known” dataset. A second “testing” phase follows in which the performance of the trained machine is assessed. The purpose of this note is to call attention to one partic-