The accuracy and precision of segmentations of medical images has been difficult to quantify in the absence of a ‘ground truth’ or reference standard segmentation for clinical data. Although physical or digital phantoms can help by providing a reference standard, they do not allow the reproduction of the full range of imaging and anatomical characteristics observed in clinical data. An alternative assessment approach is to compare with segmentations generated by domain experts. Segmentations may be generated by raters who are trained experts or by automated image analysis algorithms. Typically, these segmentations differ due to intra-rater and inter-rater variability. The most appropriate way to compare such segmentations has been unclear. We present here a new algorithm to enable the estimation of performance characteristics, and a true labelling, from observations of segmentations of imaging data where segmentation labels may be ordered or continuous measures. This approach may be used with, among others, surface, distance transform or level-set representations of segmentations, and can be used to assess whether or not a rater consistently overestimates or underestimates the position of a boundary.
[1]
Torsten Rohlfing,et al.
Expectation Maximization Strategies for Multi-atlas Multi-label Segmentation
,
2003,
IPMI.
[2]
G. McLachlan,et al.
The EM algorithm and extensions
,
1996
.
[3]
William M. Wells,et al.
Validation of Image Segmentation and Expert Quality with an Expectation-Maximization Algorithm
,
2002,
MICCAI.
[4]
William M. Wells,et al.
Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation
,
2004,
IEEE Transactions on Medical Imaging.
[5]
Torsten Rohlfing,et al.
Performance-based classifier combination in atlas-based image segmentation using expectation-maximization parameter estimation
,
2004,
IEEE Transactions on Medical Imaging.
[6]
L. R. Dice.
Measures of the Amount of Ecologic Association Between Species
,
1945
.
[7]
William M. Wells,et al.
Validation of Image Segmentation by Estimating Rater Bias and Variance
,
2006,
MICCAI.
[8]
D. Rubin,et al.
Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper
,
1977
.