Automated tongue diagnosis on the smartphone and its applications

Tongue features are important objective basis for clinical diagnosis and treatment in both western medicine and Chinese medicine. The need for continuous monitoring of health conditions inspires us to develop an automatic tongue diagnosis system based on built-in sensors of smartphones. However, tongue images taken by smartphone are quite different in color due to various lighting conditions, and it consequently affects the diagnosis especially when we use the appearance of tongue fur to infer health conditions. In this paper, we captured paired tongue images with and without flash, and the color difference between the paired images is used to estimate the lighting condition based on the Support Vector Machine (SVM). The color correction matrices for three kinds of common lights (i.e., fluorescent, halogen and incandescent) are pre-trained by using a ColorChecker-based method, and the corresponding pre-trained matrix for the estimated lighting is then applied to eliminate the effect of color distortion. We further use tongue fur detection as an example to discuss the effect of different model parameters and ColorCheckers for training the tongue color correction matrix under different lighting conditions. Finally, in order to demonstrate the potential use of our proposed system, we recruited 246 patients over a period of 2.5 years from a local hospital in Taiwan and examined the correlations between the captured tongue features and alanine aminotransferase (ALT)/aspartate aminotransferase (AST), which are important bio-markers for liver diseases. We found that some tongue features have strong correlation with AST or ALT, which suggests the possible use of these tongue features captured on a smartphone to provide an early warning of liver diseases.

[1]  John Y. Chiang,et al.  The Study on the Agreement between Automatic Tongue Diagnosis System and Traditional Chinese Medicine Practitioners , 2012, Evidence-based complementary and alternative medicine : eCAM.

[2]  Michael R Hamblin,et al.  CA : A Cancer Journal for Clinicians , 2011 .

[3]  Ratchadaporn Kanawong,et al.  ZHENG classification in Traditional Chinese Medicine based on modified specular-free tongue images , 2012, 2012 IEEE International Conference on Bioinformatics and Biomedicine Workshops.

[4]  Ye Duan,et al.  Computer-aided tongue image diagnosis and analysis , 2012 .

[5]  Ying-Ling Chen,et al.  Automatic Tongue Diagnosis System , 2009, 2009 2nd International Conference on Biomedical Engineering and Informatics.

[6]  Yu-Cheng Hsu,et al.  Automatic tongue feature extraction , 2010, 2010 International Computer Symposium (ICS2010).

[7]  Kun-Chan Lan,et al.  Color Correction Parameter Estimation on the Smartphone and Its Application to Automatic Tongue Diagnosis , 2015, Journal of Medical Systems.

[8]  Bo Peng,et al.  An SA-GA-BP neural network-based color correction algorithm for TCM tongue images , 2014, Neurocomputing.

[9]  Zhang Xin-feng.  Application of the Multi-class SVM to the Classification and the Recognition of Tongue Substance and Tongue Coat , 2004 .

[10]  Itiro Siio,et al.  TongueDx: a tongue diagnosis for health care on smartphones , 2014, AH.

[11]  Cai Yiheng,et al.  Recent progress in new portable device for tongue image analysis , 2008, 2008 International Conference on Neural Networks and Signal Processing.

[12]  张永红,et al.  Automatic Extraction of Tongue Coatings from Digital Images： A Traditional Chinese Medicine Diagnostic Tool , 2009 .

[13]  Shen Lan.  Image Analysis for Tongue Characterization , 2001 .

[14]  Yong-Feng Zhang,et al.  Study on relationship between the thickness of tongue fur and the expressions of apoptosis-related genes of the tongue epithelial cells in patients with diseases of the digestive system. , 2007, Journal of traditional Chinese medicine = Chung i tsa chih ying wen pan.

[15]  David Zhang,et al.  Tongue coating image retrieval , 2011, 2011 3rd International Conference on Advanced Computer Control.

[16]  Junwen Zhang,et al.  Extraction of tongue feature related to TCM physique based on image processing , 2015, 2015 12th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP).

[17]  Bo Huang,et al.  Quantified Vector Oriented Tongue Color Classification , 2009, 2009 2nd International Conference on Biomedical Engineering and Informatics.

[18]  Xipeng Qiu,et al.  A medical image color correction method base on supervised color constancy , 2008, 2008 IEEE International Symposium on IT in Medicine and Education.

[19]  Hong Wang,et al.  Research on the relationship between thick greasy tongue fur formation and vascular endothelial cell permeability with the protein expression of zonula occludens-1 , 2011, Chinese journal of integrative medicine.

[20]  David Zhang,et al.  A New Tongue Colorchecker Design by Space Representation for Precise Correction , 2013, IEEE Journal of Biomedical and Health Informatics.

[21]  David Zhang,et al.  A high quality color imaging system for computerized tongue image analysis , 2013, Expert Syst. Appl..

[22]  Hong Cheng,et al.  A Novel Correction Algorithm Based on Polynomial and TPS Models , 2011, 2011 International Conference of Information Technology, Computer Engineering and Management Sciences.

[23]  D. Zhang,et al.  Computer Aided Tongue Diagnosis System , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[24]  Jing Li,et al.  [Immunological mechanism of exfoliative tongue fur in children with asthma]. , 2005, Zhong xi yi jie he xue bao = Journal of Chinese integrative medicine.

[25]  Zhifeng Zhang,et al.  The region partition of quality and coating for tongue image based on color image segmentation method , 2008, 2008 IEEE International Symposium on IT in Medicine and Education.

[26]  Jingjing Wang,et al.  Classification of tongue coating using Gabor and Tamura features on unbalanced data set , 2013, 2013 IEEE International Conference on Bioinformatics and Biomedicine.

[27]  Vincenzo Savarino,et al.  Liver enzyme alteration: a guide for clinicians , 2005, Canadian Medical Association Journal.

[28]  C. H. Wang,et al.  Using threshold method to separate the edge, coating and body of tongue in automatic tongue diagnosis , 2010, The 6th International Conference on Networked Computing and Advanced Information Management.

[29]  Yang Cai,et al.  A novel imaging system for tongue inspection , 2002, IMTC/2002. Proceedings of the 19th IEEE Instrumentation and Measurement Technology Conference (IEEE Cat. No.00CH37276).

[30]  Qun Chen,et al.  A new pattern recognition algorithm and its application about tongue fur image classification of Traditional Chinese Medicine , 2010, 2010 3rd International Conference on Biomedical Engineering and Informatics.

[31]  Dong Xu,et al.  An automatic tongue detection and segmentation framework for computer-aided tongue image analysis , 2011, 2011 IEEE 13th International Conference on e-Health Networking, Applications and Services.

[32]  Yan Liu,et al.  Objective tongue inspection on 142 liver cancer patients with damp-heat syndrome , 2014, Chinese Journal of Integrative Medicine.

[33]  Xiao-Qiang Yue,et al.  [Quantitative study on tongue color in primary liver cancer patients by analysis system for comprehensive information of tongue diagnosis]. , 2003, Zhong xi yi jie he xue bao = Journal of Chinese integrative medicine.

[34]  F. Lozano.  Basic Theories of Traditional Chinese Medicine , 2014 .

[35]  A. Jemal,et al.  Global cancer statistics , 2011, CA: a cancer journal for clinicians.

[36]  Du Jian-qiang,et al.  A Novel Approach of Tongue Body and Tongue Coating Separation Based on FCM , 2008, 2008 2nd International Conference on Bioinformatics and Biomedical Engineering.

[37]  David Zhang,et al.  SVR based color calibration for tongue image , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[38]  T. Smith,et al.  The C.I.E. colorimetric standards and their use , 1931 .

[39]  Lansun Shen,et al.  Research on color constancy under open illumination conditions , 2009 .

[40]  Jane You,et al.  Tongue Color Analysis for Medical Application , 2013, Evidence-based complementary and alternative medicine : eCAM.

[41]  K.H. Kim,et al.  Tongue diagnosis method for extraction of effective region and classification of tongue coating , 2008, 2008 First Workshops on Image Processing Theory, Tools and Applications.

[42]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[43]  Wei-Kang Wu,et al.  [Identification of salivary biomarkers in breast cancer patients with thick white or thick yellow tongue fur using isobaric tags for relative and absolute quantitative proteomics]. , 2011, Zhong xi yi jie he xue bao = Journal of Chinese integrative medicine.

[44]  David Zhang,et al.  An Optimized Tongue Image Color Correction Scheme , 2010, IEEE Transactions on Information Technology in Biomedicine.

[45]  M. Hazelton.  Variable kernel density estimation , 2003 .

[46]  Wenshu Li,et al.  The separation framework of tongue coating and proper in Traditional Chinese Medicine , 2009, 2009 7th International Conference on Information, Communications and Signal Processing (ICICS).

[47]  David Zhang,et al.  Statistical Analysis of Tongue Images for Feature Extraction and Diagnostics , 2013, IEEE Transactions on Image Processing.

[48]  Weitong Huang,et al.  Analysis of the tongue fur and tongue features by naive Bayesian classifier , 2010, 2010 International Conference on Computer Application and System Modeling (ICCASM 2010).

[49]  Duan-Yu Chen,et al.  Region-based hierarchical tongue feature extraction , 2014, 2014 International Conference on Machine Learning and Cybernetics.