Cloud-based intelligent self-diagnosis and department recommendation service using Chinese medical BERT

With the rapid development of hospital informatization and Internet medical service in recent years, most hospitals have launched online hospital appointment registration systems to remove patient queues and improve the efficiency of medical services. However, most of the patients lack professional medical knowledge and have no idea of how to choose department when registering. To instruct the patients to seek medical care and register effectively, we proposed CIDRS, an intelligent self-diagnosis and department recommendation framework based on Chinese medical Bidirectional Encoder Representations from Transformers (BERT) in the cloud computing environment. We also established a Chinese BERT model (CHMBERT) trained on a large-scale Chinese medical text corpus. This model was used to optimize self-diagnosis and department recommendation tasks. To solve the limited computing power of terminals, we deployed the proposed framework in a cloud computing environment based on container and micro-service technologies. Real-world medical datasets from hospitals were used in the experiments, and results showed that the proposed model was superior to the traditional deep learning models and other pre-trained language models in terms of performance.

[1]  Stephen S. Yau,et al.  Towards Green Service Composition Approach in the Cloud , 2018, IEEE Transactions on Services Computing.

[2]  Xuyun Zhang,et al.  A balanced virtual machine scheduling method for energy-performance trade-offs in cyber-physical cloud systems , 2017, Future Gener. Comput. Syst..

[3]  Chang Liu,et al.  A cloud-based framework for Home-diagnosis service over big medical data , 2015, J. Syst. Softw..

[4]  Xiaolong Xu,et al.  Efficient computation offloading for Internet of Vehicles in edge computing-assisted 5G networks , 2019, The Journal of Supercomputing.

[5]  Qiang He,et al.  An IoT-Oriented data placement method with privacy preservation in cloud environment , 2018, J. Netw. Comput. Appl..

[6]  Jin Sun,et al.  Improving Availability of Multicore Real-Time Systems Suffering Both Permanent and Transient Faults , 2019, IEEE Transactions on Computers.

[7]  Ching-Hsien Hsu,et al.  Edge server placement in mobile edge computing , 2019, J. Parallel Distributed Comput..

[8]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[9]  Md Zakirul Alam Bhuiyan,et al.  A Dual Privacy Preserving Scheme in Continuous Location-Based Services , 2018, IEEE Internet of Things Journal.

[10]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[11]  Yiming Yang,et al.  XLNet: Generalized Autoregressive Pretraining for Language Understanding , 2019, NeurIPS.

[12]  Hao Tian,et al.  ERNIE 2.0: A Continual Pre-training Framework for Language Understanding , 2019, AAAI.

[13]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[14]  Junlong Zhou,et al.  Cost and makespan-aware workflow scheduling in hybrid clouds , 2019, J. Syst. Archit..

[15]  Lianyong Qi,et al.  Keywords-Driven and Popularity-Aware Paper Recommendation Based on Undirected Paper Citation Graph , 2020, Complex..

[16]  Hilde van der Togt,et al.  Publisher's Note , 2003, J. Netw. Comput. Appl..

[17]  Shaobo Zhang,et al.  A caching and spatial K-anonymity driven privacy enhancement scheme in continuous location-based services , 2019, Future Gener. Comput. Syst..

[18]  Xuyun Zhang,et al.  Privacy-Aware Data Fusion and Prediction With Spatial-Temporal Context for Smart City Industrial Environment , 2021, IEEE Transactions on Industrial Informatics.

[19]  Qin Liu,et al.  A Dual Privacy Preserving Scheme in Continuous Location-Based Services , 2017, 2017 IEEE Trustcom/BigDataSE/ICESS.

[20]  Xuyun Zhang,et al.  BeCome: Blockchain-Enabled Computation Offloading for IoT in Mobile Edge Computing , 2020, IEEE Transactions on Industrial Informatics.

[21]  W. Chapman,et al.  Chief Complaints and ICD Codes , 2006, Handbook of Biosurveillance.

[22]  Nan Yang,et al.  A disease diagnosis and treatment recommendation system based on big data mining and cloud computing , 2018, Inf. Sci..

[23]  Yi Pan,et al.  Automatic ICD-9 coding via deep transfer learning , 2019, Neurocomputing.

[24]  Fei Li,et al.  ICD Coding from Clinical Text Using Multi-Filter Residual Convolutional Neural Network , 2019, AAAI.

[25]  Guoyin Wang,et al.  Joint Embedding of Words and Labels for Text Classification , 2018, ACL.

[26]  Xuyun Zhang,et al.  Multi-dimensional quality-driven service recommendation with privacy-preservation in mobile edge environment , 2020, Comput. Commun..

[27]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[28]  Gerhard Weikum,et al.  Fast logistic regression for text categorization with variable-length n-grams , 2008, KDD.

[29]  Daling Wang,et al.  Context-Aware Chinese Microblog Sentiment Classification with Bidirectional LSTM , 2016, APWeb.

[30]  Jaewoo Kang,et al.  BioBERT: a pre-trained biomedical language representation model for biomedical text mining , 2019, Bioinform..

[31]  Pengtao Xie,et al.  A Neural Architecture for Automated ICD Coding , 2017, ACL.

[32]  Yuan Xue,et al.  Joint Optimization of Energy Conservation and Migration Cost for Complex Systems in Edge Computing , 2019, Complex..

[33]  Jimeng Sun,et al.  Explainable Prediction of Medical Codes from Clinical Text , 2018, NAACL.

[34]  Jun Zhao,et al.  Recurrent Convolutional Neural Networks for Text Classification , 2015, AAAI.

[35]  Kim-Kwang Raymond Choo,et al.  Enhancing privacy through uniform grid and caching in location-based services , 2017, Future Gener. Comput. Syst..

[36]  Wanchun Dou,et al.  Privacy-Aware Cross-Platform Service Recommendation Based on Enhanced Locality-Sensitive Hashing , 2021, IEEE Transactions on Network Science and Engineering.

[37]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[38]  Xuyun Zhang,et al.  Finding All You Need: Web APIs Recommendation in Web of Things Through Keywords Search , 2019, IEEE Transactions on Computational Social Systems.

[39]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[40]  Iz Beltagy,et al.  SciBERT: A Pretrained Language Model for Scientific Text , 2019, EMNLP.

[41]  Ching-Hsien Hsu,et al.  Service Composition in Cyber-Physical-Social Systems , 2020, IEEE Transactions on Emerging Topics in Computing.

[42]  Fei Dai,et al.  Dynamic Resource Provisioning With Fault Tolerance for Data-Intensive Meteorological Workflows in Cloud , 2020, IEEE Transactions on Industrial Informatics.

[43]  Tong Zhang,et al.  Deep Pyramid Convolutional Neural Networks for Text Categorization , 2017, ACL.

[44]  Junlong Zhou,et al.  Security-Critical Energy-Aware Task Scheduling for Heterogeneous Real-Time MPSoCs in IoT , 2020, IEEE Transactions on Services Computing.