Automatically Localizing a Large Set of Spatially Correlated Key Points: A Case Study in Spine Imaging

The fully automatic localization of key points in medical images is an important and active area in applied machine learning, with very large sets of key points still being an open problem. To this end, we extend two general state-of-the-art localization approaches to operate on large amounts of key points and evaluate both approaches on a CT spine data set featuring 102 key points. First, we adapt the multi-stage convolutional pose machines neural network architecture to 3D image data with some architectural changes to cope with the large amount of data and key points. Imprecise localizations caused by the inherent downsampling of the network are countered by quadratic interpolation. Second, we extend a common approach—regression tree ensembles spatially regularized by a conditional random field—by a latent scaling variable to explicitly model spinal size variability. Both approaches are evaluated in detail in a 5-fold cross-validation setup in terms of localization accuracy and test time on 157 spine CT images. The best configuration achieves a mean localization error of 4.21 mm over all 102 key points.