The Lung Image Database Consortium (LIDC) data collection process for nodule detection and annotation

The LIDC is developing a publicly available database of thoracic computed tomography (CT) scans as a medical imaging research resource. A unique multi-center data collection process and communication system were developed to share image data and to capture the location and spatial extent of lung nodules as marked by expert radiologists. A two-phase data collection process was designed to allow multiple radiologists at different centers to asynchronously review and annotate each CT image series. Four radiologists reviewed each case using this process. In the first or "blinded" phase, each radiologist reviewed the CT series independently. In the second or "unblinded" review phase, the results from all four blinded reviews are compiled and presented to each radiologist for a second review. This allows each radiologist to review their own annotations along with those of the other radiologists. The results from each radiologist's unblinded review were compiled to form the final unblinded review. There is no forced consensus in this process. An XML-based message system was developed to communicate the results of each reading. This two-phase data collection process was designed, tested and implemented across the LIDC. It has been used for more than 130 CT cases that have been read and annotated by four expert readers and are publicly available at (http://ncia.nci.nih.gov). A data collection process was developed, tested and implemented that allowed multiple readers to review each case multiple times and that allowed each reader to observe the annotations of other readers.