Semi-AI and Full-AI digitizer: The ways to digitalize visual field big data

BACKGROUND AND OBJECTIVE Glaucoma is one of the major diseases that cause blindness, which is incurable and irreversible, and it is essential to detect glaucoma vision deficits in treatment and check the progression of vision disorders in advance. In order to minimize the risk of glaucoma, it is necessary not only to diagnose and observe glaucoma but also to predict prognosis via indicators from Visual Field (VF) tests. However, information from the VF test cannot be directly used in clinical studies because most medical institutions store VF test sheets in Portable Document Format (PDF) or image files in different standards. METHODS We developed AI-based real-time VF big data digitizing systems that digitalize VF test images in real-time in two ways; Semi-AI and Full-AI digitizer. The Semi-AI digitizer detects the VF text area with actual coordinates derived from mouse handler system. Full-AI digitizer detects the VF text area with Faster Region Based Convolutional Neural Networks (RCNN). After detecting the text area, both systems extract texts with Recurrent Neural Network based Optical Character Recognition. Semi-AI and Full-AI digitizer post-processes the extracted text results with in-system algorithm and out-of-system algorithm, respectively. RESULTS Both systems used 325,310 VF test sheets from a tertiary hospital and extracted a total of 5,530,270 texts. From the 100 randomly selected VF sheets, 3,400 texts were used for the validation. Semi-AI and Full-AI digitizer showed 0.993 and 0.983 of accuracy, respectively. CONCLUSION This study demonstrates the effectiveness of AI applications in detecting text areas and the different implementation methodologies of the post-processing process. In detecting text area, Semi-AI may be better than Full-AI digitizer in terms of system speed and human labor labeling if the number of types to be classified is small. However, Full-AI digitizer is recommended because it allows detecting text area regardless of resolution and size of the VF sheets, as the types of real-world VF test sheets cannot be predicted, and the types become more unpredictable when extended to multi-hospital studies. For Post-preprocessing, Semi-AI methodology is recommended because Semi-AI produced higher results with less effort and considered the convenience of researchers by implementing them as in-system.

[1]  A. Sommer,et al.  The nerve fiber layer in the diagnosis of glaucoma. , 1977, Archives of ophthalmology.

[2]  Mirco Fabbri,et al.  Dow Jones Trading with Deep Learning: The Unreasonable Effectiveness of Recurrent Neural Networks , 2018, DATA.

[3]  Dong Yoon Kim,et al.  Clinical Validation of Visual Field Index , 2010 .

[4]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Terrence J. Sejnowski,et al.  Comparison of machine learning and traditional classifiers in glaucoma diagnosis , 2002, IEEE Transactions on Biomedical Engineering.

[6]  Patrice Y. Simard,et al.  Best practices for convolutional neural networks applied to visual document analysis , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[7]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Deep learning for biological image classification , 2017, Expert Syst. Appl..

[8]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[9]  P. Mildenberger,et al.  Introduction to the DICOM standard , 2002, European Radiology.

[10]  Yue Wu,et al.  Forecasting future Humphrey Visual Fields using deep learning , 2018, bioRxiv.

[11]  Advanced Glaucoma Intervention Study. 2. Visual field test scoring and reliability. , 1994, Ophthalmology.

[12]  Andrew Y. Ng,et al.  Parsing Natural Scenes and Natural Language with Recursive Neural Networks , 2011, ICML.

[13]  Oscar Deniz Suarez,et al.  Learning Image Processing with OpenCV , 2015 .

[14]  E. Goceri,et al.  Deep learning based classification of facial dermatological disorders , 2020, Comput. Biol. Medicine.

[15]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  D. Budenz.  Atlas of Visual Fields , 1997 .

[18]  In Young Choi,et al.  CIMI: Classify and Itemize Medical Image System for PFT Big Data Based on Deep Learning , 2020 .

[19]  Christian Riess,et al.  A Gentle Introduction to Deep Learning in Medical Image Processing , 2018, Zeitschrift fur medizinische Physik.

[20]  Muhammad Imran Razzak,et al.  Deep Learning for Medical Image Processing: Overview, Challenges and Future , 2017, ArXiv.

[21]  Ching Y. Suen,et al.  The State of the Art in Online Handwriting Recognition , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Evgin Goceri,et al.  CapsNet topology to classify tumours from brain images and comparative evaluation , 2020, IET Image Process..

[23]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[24]  W. Bieniecki,et al.  Image Preprocessing for Improving OCR Accuracy , 2007, 2007 International Conference on Perspective Technologies and Methods in MEMS Design.

[25]  Evgin Goceri,et al.  Diagnosis of Alzheimer's disease with Sobolev gradient‐based optimization and 3D convolutional neural network , 2019, International journal for numerical methods in biomedical engineering.

[26]  R. Hitchings,et al.  The optic disc in glaucoma II: correlation of the appearance of the optic disc with the visual field. , 1977, The British journal of ophthalmology.

[27]  Sooyoung Yoo,et al.  An innovative strategy for standardized, structured, and interoperable results in ophthalmic examinations , 2021, BMC Medical Informatics and Decision Making.

[28]  Andreas Dengel,et al.  Two-stage framework for optic disc localization and glaucoma classification in retinal fundus images using deep learning , 2019, BMC Medical Informatics and Decision Making.

[29]  Evgin Göçeri.  Convolutional Neural Network Based Desktop Applications to Classify Dermatological Diseases , 2020, 2020 IEEE 4th International Conference on Image Processing, Applications and Systems (IPAS).

[30]  Patel Dhruv,et al.  Image Classification Using Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN): A Review , 2020 .

[31]  Gang Li,et al.  Predictive factors for glaucomatous visual field progression in the Advanced Glaucoma Intervention Study. , 2004, Ophthalmology.

[32]  B. Bengtsson,et al.  A visual field index for calculation of glaucoma rate of progression. , 2008, American journal of ophthalmology.

[33]  Jonghoon Shin,et al.  Prediction of visual field from swept-source optical coherence tomography using deep learning algorithms , 2020, Graefe's Archive for Clinical and Experimental Ophthalmology.