Public database for validation of follicle detection algorithms on 3D ultrasound images of ovaries

BACKGROUND AND OBJECTIVE Automated follicle detection in ovarian ultrasound volumes remains a challenging task. An objective comparison of different follicle-detection approaches is only possible when all are tested on the same data. This paper describes the development and structure of the first publicly accessible USOVA3D database of annotated ultrasound volumes with ovarian follicles. METHODS The ovary and all follicles were annotated in each volume by two medical experts. The USOVA3D database is supplemented by a general verification protocol for unbiased assessment of detection algorithms that can be compared and ranked by scoring according to this protocol. This paper also introduces two baseline automated follicle-detection algorithms, the first based on Directional 3D Wavelet Transform (3D DWT) and the second based on Convolutional Neural Networks (CNN). RESULTS The USOVA3D testing data set was used to verify the variability and reliability of follicle annotations. The intra-rater overall score yielded around 83 (out of a maximum of 100), while both baseline algorithms pointed out just a slightly lower performance, with the 3D DWT-based algorithm being better, with an overall score around 78. CONCLUSIONS On the other hand, the development of the CNN-based algorithm demonstrated that the USOVA3D database contains sufficient data for successful training without overfitting. The inter-rater reliability analysis and the obtained statistical metrics of effectiveness for both baseline algorithms confirmed that the USOVA3D database is a reliable source for developing new automated detection methods.