Extract Bone Parts Without Human Prior: End-to-end Convolutional Neural Network for Pediatric Bone Age Assessment

Pediatric bone age assessment (BAA) is a common clinical practice to investigate endocrinology, genetic and growth disorders of children. The morphological characters of different specific bone parts, such as wrist and phalanx, have important reference significance in BAA. Previous deep learning approaches can be divided into two branches, (1) the single-stage structure ignores the attention on specific bone parts, thus it can be trained end-to-end but suffers from low accuracy, (2) the multi-stage structure extracts the bone parts with human prior, thus it exhibits high accuracy but suffers from model generalization and resource consumption problem. To enable an end-to-end training method extracting discriminative bone parts automatically without human prior, in this paper, we propose a novel single-stage Attention-Recognition Convolutional Neural Network (AR-CNN). The AR-CNN consists of one attention agent for discriminative bone parts proposing and one recognition agent for feature learning and age assessment. The attention agent can discover and extract bone parts automatically, meanwhile the recognition agent can learn the features from the proposing bone parts and assess the bone age. Furthermore, the assessment result will be fed back to attention agent for the optimization of bone parts extracting. Therefore, the two agents can reinforce each other mutually and the overall network can be trained end-to-end without human prior. To the best of our knowledge, this is the first end-to-end structure to extract bone parts for BAA without segmentation, detection and human prior. Experimental results show that our approach achieves state-of-the-art accuracy on the public RSNA datasets with mean absolute error(MAE) of 4.38 months.