Multi-Granularity Tooth Analysis via Faster Region-Convolutional Neural Networks for Effective Tooth Detection and Classification

—In image classification, multi-granularity refers to the ability to classify images with different levels of detail or resolution. This is a challenging task because the distinction between subcategories is often minimal, needing a high level of visual detail and precise representation of the features specific to each class. In dental informatics, and more specifically tooth classification poses many challenges due to overlapping teeth, varying sizes, shapes, and illumination levels. To address these issues, this paper considers various data granularity levels since a deeper level of details can be acquired with increased granularity. Three tooth granularity levels are considered in this study named Two Classes Granularity Level (2CGL), Four Classes Granularity Level (4CGL), and Seven Classes Granularity Level (7CGL) to analyze the performance of teeth detection and classification at multi-granularity levels in Granular Intra-Oral Image (GIOI) dataset. Subsequently, a Faster Region-Convolutional Neural Network (FR-CNN) based on three ResNet models is proposed for teeth detection and classification at multi-granularity levels from the GIOI dataset. The FR-CNN-ResNet models exploit the effect of the tooth classification granularity technique to empower the models with accurate features that lead to improved model performance. The results indicate a remarkable detection effect in investigating the granularity effect on the FR-CNN-ResNet model's performance. The FR-CNN-ResNet-50 model achieved 0.94 mAP for 2CGL, 0.74 mAP for 4CGL, and 0.69 mAP for 7CGL, respectively. The findings demonstrated that multi-granularity enables flexible and nuanced analysis of visual data, which can be useful in a wide range of applications.

[1]  E. M. Canger,et al.  Developing deep learning methods for classification of teeth in dental panoramic radiography. , 2023, Oral surgery, oral medicine, oral pathology and oral radiology.

[2]  Donghai Zhai,et al.  Multi-Granularity Dilated Transformer for Lung Nodule Classification via Local Focus Scheme , 2022, Applied Sciences.

[3]  Namrata Ansari,et al.  A Survey on Dental Disease Detection Based on Deep Learning Algorithm Performance using Various Radiographs , 2022, 2022 5th International Conference on Advances in Science and Technology (ICAST).

[4]  Samah W. G. AbuSalim,et al.  Data Augmentation on Intra-Oral Images Using Image Manipulation Techniques , 2022, 2022 International Conference on Digital Transformation and Intelligence (ICDI).

[5]  H. Benzian,et al.  WHO calls to end the global crisis of oral health , 2022, The Lancet.

[6]  Lequan Yu,et al.  Multi-Granularity Cross-modal Alignment for Generalized Medical Visual Representation Learning , 2022, NeurIPS.

[7]  Samah W. G. AbuSalim,et al.  Analysis of Deep Learning Techniques for Dental Informatics: A Systematic Literature Review , 2022, Healthcare.

[8]  Jifeng Shen,et al.  Multi-Granularity Feature Aggregation with Self-Attention and Spatial Reasoning for Fine-Grained Crop Disease Classification , 2022, Agriculture.

[9]  Jinqiao Wang,et al.  Multi-Granularity Mutual Learning Network for Object Re-Identification , 2022, IEEE Transactions on Intelligent Transportation Systems.

[10]  Y. Ye,et al.  A Combined Approach for Accurate and Accelerated Teeth Detection on Cone Beam CT Images , 2022, Diagnostics.

[11]  T. Tanprasert,et al.  Image classification based on multi-granularity convolutional Neural network model , 2022, 2022 19th International Joint Conference on Computer Science and Software Engineering (JCSSE).

[12]  Pengfei Zhu,et al.  Multi-granularity episodic contrastive learning for few-shot learning , 2022, Pattern Recognit..

[13]  Jia-Bin Huang,et al.  A fully automatic AI system for tooth and alveolar bone segmentation from cone-beam CT images , 2022, Nature Communications.

[14]  Ning Dai,et al.  A Dual Discriminator Adversarial Learning Approach for Dental Occlusal Surface Reconstruction , 2022, Journal of healthcare engineering.

[15]  Muhammad Jamal Ahmed,et al.  A robust approach for industrial small-object detection using an improved faster regional convolutional neural network , 2021, Scientific Reports.

[16]  S. Weinberg,et al.  Dental anomaly detection using intraoral photos via deep learning , 2021, Scientific Reports.

[17]  Jing Chen,et al.  IMAGE DETECTION OF DENTAL DISEASES BASED ON DEEP TRANSFER LEARNING , 2021, 2021 2nd International Conference on Artificial Intelligence and Computer Engineering (ICAICE).

[18]  D. Gebauer,et al.  Deep learning for automated detection and numbering of permanent teeth on panoramic images. , 2021, Dento maxillo facial radiology.

[19]  I. Bayrakdar,et al.  Performance of a convolutional neural network algorithm for tooth detection and numbering on periapical radiographs. , 2021, Dento maxillo facial radiology.

[20]  S. Sukaridhoto,et al.  Smart Odontogram: Dental Diagnosis of Patients Using Deep Learning , 2021, 2021 International Electronics Symposium (IES).

[21]  Anuj Kumar,et al.  Descriptive analysis of dental X-ray images using various practical methods: A review , 2021, PeerJ Comput. Sci..

[22]  R. Hickel,et al.  Caries Detection on Intraoral Images Using Artificial Intelligence , 2021, Journal of dental research.

[23]  Guoyin Wang,et al.  Multi-Granularity Neural Network Encoding Method for Land Cover and Land Use Image Classification , 2021 .

[24]  I. Różyło-Kalinowska,et al.  An artifıcial ıntelligence approach to automatic tooth detection and numbering in panoramic radiographs , 2021, BMC Medical Imaging.

[25]  Amjad J. Humaidi,et al.  Review of deep learning: concepts, CNN architectures, challenges, applications, future directions , 2021, Journal of Big Data.

[26]  Dhruv Verma,et al.  Anomaly detection in panoramic dental x-rays using a hybrid Deep Learning and Machine Learning approach , 2020, 2020 IEEE REGION 10 CONFERENCE (TENCON).

[27]  I. Bayrakdar,et al.  An artificial intelligence proposal to automatic teeth detection and numbering in dental bite-wing radiographs , 2020, Acta odontologica Scandinavica.

[28]  Syoji Kobashi,et al.  Optimization technique combined with deep learning method for teeth recognition in dental panoramic radiographs , 2020, Scientific Reports.

[29]  Yi Xiao,et al.  Object Detection Based on Faster R-CNN Algorithm with Skip Pooling and Fusion of Contextual Information , 2020, Sensors.

[30]  Siriwan Suebnukarn,et al.  Automatic detection of periodontal compromised teeth in digital panoramic radiographs using faster regional convolutional neural networks , 2020, Imaging science in dentistry.

[31]  Haitao Guo,et al.  An Efficient and Lightweight Convolutional Neural Network for Remote Sensing Image Scene Classification , 2020, Sensors.

[32]  Gloria Hyunjung Kwak,et al.  Automatic mandibular canal detection using a deep convolutional neural network , 2020, Scientific Reports.

[33]  A. Katsumata,et al.  Tooth detection and classification on panoramic radiographs for automatic dental chart filing: improved classification by multi-sized input data , 2020, Oral Radiology.

[34]  A. Katsumata,et al.  Tooth detection and classification on panoramic radiographs for automatic dental chart filing: improved classification by multi-sized input data , 2020, Oral Radiology.

[35]  F. Schwendicke,et al.  Convolutional neural networks for dental image diagnostics: A scoping review. , 2019, Journal of dentistry.

[36]  Xu Yan,et al.  An Improved Faster R-CNN for Small Object Detection , 2019, IEEE Access.

[37]  Taghi M. Khoshgoftaar,et al.  A survey on Image Data Augmentation for Deep Learning , 2019, Journal of Big Data.

[38]  Zhiming Cui,et al.  ToothNet: Automatic Tooth Instance Segmentation and Identification From Cone Beam CT Images , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Abhishek Dutta,et al.  The VIA Annotation Software for Images, Audio and Video , 2019, ACM Multimedia.

[40]  Hong Li,et al.  A deep learning approach to automatic teeth detection and numbering based on object detection in dental periapical films , 2019, Scientific Reports.

[41]  Michael M Bornstein,et al.  Tooth detection and numbering in panoramic radiographs using convolutional neural networks. , 2019, Dento maxillo facial radiology.

[42]  Diana Marculescu,et al.  Understanding the Impact of Label Granularity on CNN-Based Image Classification , 2018, 2018 IEEE International Conference on Data Mining Workshops (ICDMW).

[43]  Ji Wu,et al.  An effective teeth recognition method using label tree with cascade network structure , 2018, Comput. Medical Imaging Graph..

[44]  Richard K. G. Do,et al.  Convolutional neural networks: an overview and application in radiology , 2018, Insights into Imaging.

[45]  Derek Bradley,et al.  Model-based teeth reconstruction , 2016, ACM Trans. Graph..

[46]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[49]  Xiaoying Tang,et al.  Tools for multiple granularity analysis of brain MRI data for individualized image analysis , 2014, NeuroImage.

[50]  Jianhui Wu,et al.  When Multigranularity Meets Spatial–Spectral Attention: A Hybrid Transformer for Hyperspectral Image Classification , 2023, IEEE Transactions on Geoscience and Remote Sensing.

[51]  Jian Yang,et al.  Remote Sensing Image Scene Classification by Multiple Granularity Semantic Learning , 2022, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[52]  Ning Wang,et al.  Occlusion Problem-Oriented Adversarial Faster-RCNN Scheme , 2019, IEEE Access.

[53]  Xiangrong Zhou,et al.  Classification of teeth in cone-beam CT using deep convolutional neural network , 2017, Comput. Biol. Medicine.

[54]  Ayse Betul Oktay.  Tooth detection with Convolutional Neural Networks , 2017 .