Inter- and intra-observer variations in the delineation of lesions in mammograms

Many clinical and research tasks require the delineation of lesions in radiological images. There is a variety of methods available for deriving such delineations, ranging from free hand manual contouring and manual positioning of lowparameter graphical objects, to (semi-)automatic computerized segmentation methods. In this paper we investigate the impact of the chosen segmentation method on the inter-observer variability of the resulting contour. Three different methods are compared in this paper, namely (1) manual positioning of an ellipse, (2) an automatic segmentation method, coined live-segmentation, which depends on the current mouse pointer position as input information and is updated in real-time as the user hovers with the mouse over the image and (3) free form segmentation which is realized by allowing the user to pull the result of method (2) to image positions that the contour is required to pass. Each of the three methods was used by three experienced radiologists to delineate a set of 215 round breast lesion images in digital mammograms. Agreement between contours was assessed by computing the Dice coefficient. The median Dice coefficient for the ellipses placed by different readers was 0.85. The intra-reader Dice coefficient comparing ellipses and livesegmentations was 0.84, thus showing that the live-segmentation results agree with ellipse segmentations to the same extent as readers agree on the ellipse placement. Inter-observer agreement when using the live-segmentation was higher than for the ellipses (median Dice = 0.91 vs. 0.85) showing that the live-segmentation is a more reproducible alternative to the ellipse placement.