Multi-Task Convolutional Neural Network for Joint Bone Age Assessment and Ossification Center Detection from Hand Radiograph

Bone age assessment is a common clinical procedure to diagnose endocrine and metabolic disorders in children. Recently, a variety of convolutional neural network based approaches have been developed to automatically estimate bone age from hand radiographs and achieved accuracy comparable to human experts. However, most of these networks were trained end-to-end, i.e., deriving the bone age directly from the whole input hand image without knowing which regions of the image are most relevant to the task. In this work, we proposed a multi-task convolutional neural network to simultaneously estimate bone age and localize ossification centers of different phalangeal, metacarpal and carpal bones. We showed that, similar to providing attention maps, the localization of ossification centers helps the network to extract features from more meaningful regions where local appearances are closely related to the skeletal maturity. In particular, to address the problem that some ossification centers do not always appear on the hand radiographs of certain bone ages, we introduced an image-level landmark presence classification loss, in addition to the conventional pixel-level landmark localization loss, in our multi-task network framework. Experiments on public RSNA data demonstrated the effectiveness of our proposed method in the reduction of gross errors of ossification center detection, as well as the improvement of bone age assessment accuracy with the aid of ossification center detection especially when the training data size is relatively small.