Automatic construction of Chinese herbal prescription from tongue image via convolution networks and auxiliary latent therapy topics

The tongue image is an important physical information of human, it is of great importance to the diagnosis and treatment in clinical medicine. Herbal prescriptions are simple, noninvasive and low side effects, and are widely applied in China. Researches on automatic construction technology of herbal prescription based on tongue image have great significance for deep learning to explore the relevance from tongue image to herbal prescription, and can be applied to healthcare services in mobile medical system. In order to adapt to the tongue image in a variety of photographing environments and construct the herbal prescriptions, a neural network framework for prescriptions construction is designed, which includes single / double convolution channels and fully connected layers, and propose the mechanism of auxiliary therapy topic loss to model the therapy of Chinese doctors then alleviate the interference of sparse output labels to the diversity of results. The experimental data include the patient tongue images and their corresponding prescriptions from real world outpatient clinic, and the experiment results can generate the prescriptions that are close to the real samples, which verifies the feasibility of the proposed method for automatic construction of herbal prescription from tongue image. Also, provides a reference for automatic herbal prescription construction from more physical information (or integrated body information).

[1]  Jeroen S. de Bruin,et al.  Assessing the feasibility of a mobile health-supported clinical decision support system for nutritional triage in oncology outpatients using Arden Syntax , 2015, Artif. Intell. Medicine.

[2]  Michael Juntao Yuan,et al.  A novel clinical decision support algorithm for constructing complete medication histories , 2017, Comput. Methods Programs Biomed..

[3]  Fabio A. González,et al.  An unsupervised feature learning framework for basal cell carcinoma image analysis , 2015, Artif. Intell. Medicine.

[4]  Guillermo Sapiro,et al.  Online dictionary learning for sparse coding , 2009, ICML '09.

[5]  Zhaohui Wu,et al.  Knowledge discovery in traditional Chinese medicine: State of the art and perspectives , 2006, Artif. Intell. Medicine.

[6]  Ralph Deters,et al.  Mobile Medical Data Synchronization on Cloud-Powered Middleware Platform , 2016, IEEE Transactions on Services Computing.

[7]  Hao Chen,et al.  3D deeply supervised network for automated segmentation of volumetric medical images , 2017, Medical Image Anal..

[8]  Christopher C. Yang,et al.  Intelligent healthcare informatics in big data era , 2015, Artif. Intell. Medicine.

[9]  Shuangge Ma,et al.  Impact of the new health care reform on hospital expenditure in China: A case study from a pilot city , 2016 .

[10]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[11]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[12]  Dong Ni,et al.  Relational-Regularized Discriminative Sparse Learning for Alzheimer’s Disease Diagnosis , 2017, IEEE Transactions on Cybernetics.

[13]  Christopher Dainton,et al.  A review of electronic medical record keeping on mobile medical service trips in austere settings , 2017, Int. J. Medical Informatics.

[14]  Ling Zhu,et al.  Knowledge graph for TCM health preservation: Design, construction, and applications , 2017, Artif. Intell. Medicine.

[15]  Mark Steyvers,et al.  Finding scientific topics , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[16]  Xiaochun Cao,et al.  Inference With Collaborative Model for Interactive Tumor Segmentation in Medical Image Sequences , 2016, IEEE Transactions on Cybernetics.

[17]  Lin Yang,et al.  MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Xi Chen,et al.  Estimation of the Volume of the Left Ventricle From MRI Images Using Deep Neural Networks , 2017, IEEE Transactions on Cybernetics.

[19]  Feng Lin,et al.  Low-Dose CT With a Residual Encoder-Decoder Convolutional Neural Network , 2017, IEEE Transactions on Medical Imaging.

[20]  Jun Zhang,et al.  Detecting Anatomical Landmarks From Limited Medical Imaging Data Using Two-Stage Task-Oriented Deep Neural Networks , 2017, IEEE Transactions on Image Processing.

[21]  Xiaohong Jiang,et al.  An intelligent medicine recommender system framework , 2016, 2016 IEEE 11th Conference on Industrial Electronics and Applications (ICIEA).

[22]  Kevin Chen-Chuan Chang,et al.  Detecting signals of detrimental prescribing cascades from social media , 2016, Artif. Intell. Medicine.

[23]  Ronald M. Summers,et al.  Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique , 2016 .

[24]  Ping Feng,et al.  Telehealth attitudes and use among medical professionals, medical students and patients in China: A cross-sectional survey , 2017, Int. J. Medical Informatics.

[25]  Xiaoming Liu,et al.  Automatic segmentation of liver tumors from multiphase contrast-enhanced CT images based on FCNs , 2017, Artif. Intell. Medicine.

[26]  Yongcheng Li,et al.  Joint similar and specific learning for diabetes mellitus and impaired glucose regulation detection , 2017, Inf. Sci..

[27]  J Kim,et al.  A digital tongue imaging system for tongue coating evaluation in patients with oral malodour. , 2009, Oral diseases.

[28]  Li Zhuo,et al.  A K-PLSR-based color correction method for TCM tongue images under different illumination conditions , 2016, Neurocomputing.

[29]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[30]  Fei Wang,et al.  Deep learning for healthcare: review, opportunities and challenges , 2018, Briefings Bioinform..

[31]  Xu Sun,et al.  Exploration on Generating Traditional Chinese Medicine Prescription from Symptoms with an End-to-End method , 2018, ArXiv.

[32]  Pingkun Yan,et al.  Label Image Constrained Multiatlas Selection , 2015, IEEE Transactions on Cybernetics.

[33]  Wei Wang,et al.  Discovering treatment pattern in Traditional Chinese Medicine clinical cases by exploiting supervised topic model and domain knowledge , 2015, J. Biomed. Informatics.

[34]  David Dagan Feng,et al.  An Ensemble of Fine-Tuned Convolutional Neural Networks for Medical Image Classification , 2017, IEEE Journal of Biomedical and Health Informatics.

[35]  Zafer D. Özdemir,et al.  Drug prescription behavior and decision support systems , 2014, Decis. Support Syst..

[36]  Bo Peng,et al.  An SA-GA-BP neural network-based color correction algorithm for TCM tongue images , 2014, Neurocomputing.

[37]  T. Kaptchuk,et al.  .‫املقالة‬ ‫لهذه‬ ‫الكامل‬ ‫النص‬ ‫نهاية‬ ‫يف‬ ‫الخالصة‬ ‫لهذه‬ ‫العربية‬ ‫الرتجمة‬ Herbal Medicine Research and Global Health: an Ethical Analysis , 2022 .

[38]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[39]  David Zhang,et al.  Computerized tongue diagnosis based on Bayesian networks , 2004, IEEE Transactions on Biomedical Engineering.

[40]  David Zhang,et al.  Statistical Analysis of Tongue Images for Feature Extraction and Diagnostics , 2013, IEEE Transactions on Image Processing.

[41]  Maciej Zieba,et al.  Service-Oriented Medical System for Supporting Decisions With Missing and Imbalanced Data , 2014, IEEE Journal of Biomedical and Health Informatics.

[42]  Zhi Liu,et al.  Tongue fissure extraction and classification using hyperspectral imaging technology. , 2010, Applied optics.

[43]  Peter Xiaoping Liu,et al.  A New Deformation Model of Biological Tissue for Surgery Simulation , 2017, IEEE Transactions on Cybernetics.

[44]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Hongying Liu,et al.  AOTF based hyperspectral tongue imaging system and its applications in computer-aided tongue disease diagnosis , 2010, 2010 3rd International Conference on Biomedical Engineering and Informatics.

[46]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[48]  Yangyang Liu,et al.  Medical image classification via multiscale representation learning , 2017, Artif. Intell. Medicine.

[49]  David Zhang,et al.  Detecting Diabetes Mellitus and Nonproliferative Diabetic Retinopathy Using Tongue Color, Texture, and Geometry Features , 2014, IEEE Transactions on Biomedical Engineering.

[50]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[51]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.