Web-based fully automated cephalometric analysis by deep learning

BACKGROUND AND OBJECTIVE An accurate lateral cephalometric analysis is vital in orthodontic diagnosis. Identification of anatomic landmarks on lateral cephalograms is tedious, and errors may occur depending on the doctor's experience. Several attempts have been made to reduce this time-consuming process by automating the process through machine learning; however, they only dealt with a small amount of data from one institute. This study aims to develop a fully automated cephalometric analysis method using deep learning and a corresponding web-based application that can be used without high-specification hardware. METHODS We built our own dataset comprising 2,075 lateral cephalograms and ground truth positions of 23 landmarks from two institutes and trained a two-stage automated algorithm with a stacked hourglass deep learning model specialized for detecting landmarks in images. Additionally, a web-based application with the proposed algorithm for fully automated cephalometric analysis was developed for better accessibility regardless of the user's computer hardware, which is essential for a deep learning-based method. RESULTS The algorithm was evaluated with datasets from various devices and institutes, including a widely used open dataset and achieved 1.37 ± 1.79 mm of point-to-point errors with ground truth positions for 23 cephalometric landmarks. Based on the predicted positions, anatomical types of the subjects were automatically classified and compared with the ground truth, and the automated algorithm achieved a successful classification rate of 88.43%. CONCLUSIONS We expect that this fully automated cephalometric analysis algorithm and the web-based application can be widely used in various medical environments to save time and effort for manual marking and diagnosis.

[1]  C. Steiner.  Cephalometrics for you and me , 1953 .

[2]  S. Nishimoto,et al.  Personal Computer-Based Cephalometric Landmark Detection With Deep Learning, Using Cephalograms on the Internet , 2019, The Journal of craniofacial surgery.

[3]  R S Nanda,et al.  Cephalometric study of the dentofacial complex of North Indians. , 1969, The Angle orthodontist.

[4]  C H TWEED,et al.  The Frankfort-mandibular plane angle in orthodontic diagnosis, classification, treatment planning, and prognosis. , 1946, American journal of orthodontics and oral surgery.

[5]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[6]  Rama Chellappa,et al.  HyperFace: A Deep Multi-Task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Christian Payer,et al.  Integrating spatial configuration into heatmap regression based CNNs for landmark localization , 2019, Medical Image Anal..

[8]  Hansang Lee,et al.  Cephalometric landmark detection in dental x-ray images using convolutional neural networks , 2017, Medical Imaging.

[9]  Stefan Winkler,et al.  Deep Learning for Emotion Recognition on Small Datasets using Transfer Learning , 2015, ICMI.

[10]  D. Waring,et al.  Adaptable Landmark Localisation: Applying Model Transfer Learning to a Shape Model Matching System , 2017, MICCAI.

[11]  Georgios Tzimiropoulos,et al.  Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[12]  Martin Urschler,et al.  Integrating geometric configuration and appearance information into a unified framework for anatomical landmark localization , 2018, Medical Image Anal..

[13]  Y. Kim,et al.  Anteroposterior dysplasia indicator: an adjunct to cephalometric differential diagnosis. , 1978, American journal of orthodontics.

[14]  W. Downs,et al.  Variations in facial relationships; their significance in treatment and prognosis. , 1948, American journal of orthodontics.

[15]  Y. Kim,et al.  Overbite depth indicator with particular reference to anterior open-bite. , 1974, American journal of orthodontics.

[16]  Jia Deng,et al.  Stacked Hourglass Networks for Human Pose Estimation , 2016, ECCV.

[17]  Sandra Sudarsky,et al.  Deep learning with cinematic rendering: fine-tuning deep neural networks using photorealistic medical images , 2018, Physics in medicine and biology.

[18]  Junmo Kim,et al.  Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[19]  Bulat Ibragimov,et al.  Fully automated quantitative cephalometry using convolutional neural networks , 2017, Journal of medical imaging.

[20]  Raphaël Marée,et al.  Landmark detection in 2D bioimages for geometric morphometrics: a multi-resolution tree-based approach , 2018, Scientific Reports.

[21]  J A McNamara,et al.  A method of cephalometric evaluation. , 1984, American journal of orthodontics.

[22]  Jian Yang,et al.  FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[23]  Gang Yu,et al.  Cascaded Pyramid Network for Multi-person Pose Estimation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Qingshan Liu,et al.  Stacked Hourglass Network for Robust Facial Landmark Localisation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[25]  Chengwen Chu,et al.  Evaluation and Comparison of Anatomical Landmark Detection Methods for Cephalometric X-Ray Images: A Grand Challenge , 2015, IEEE Transactions on Medical Imaging.

[26]  Timothy F. Cootes,et al.  A benchmark for comparison of dental radiography analysis algorithms , 2016, Medical Image Anal..

[27]  Shiguang Shan,et al.  Heterogeneous Face Attribute Estimation: A Deep Multi-Task Learning Approach , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Xiaogang Wang,et al.  Learning Feature Pyramids for Human Pose Estimation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  Ching-Wei Wang,et al.  Fully Automatic System for Accurate Localisation and Analysis of Cephalometric Landmarks in Lateral Cephalograms , 2016, Scientific Reports.