Transfer learning with deep convolutional neural network for constitution classification with face image

Constitution classification is the basis and core content of constitution research in Traditional Chinese medicine. The convolutional neural networks have successfully established many models for image classification, but it requires a lot of training data. In the field of Traditional Chinese medicine, the available clinical data is very limited. To solve this problem, we propose a method for constitution classification through transfer learning. Firstly, the DenseNet-169 model trained in ImageNet is applied. Secondly, we carefully modify the DenseNet-169 structure according to the constitution characteristics, and then the modified model is trained in the clinical data to obtain the constitution identification network called ConstitutionNet. In order to further improve the accuracy of classification, we integrate the ConstitutionNet with Vgg-16, Inception v3 and DenseNet-121 to test according to the integrated learning idea, and judge the input face image to its constitution type. The experimental results show that transfer learning can achieve better results in small clinical dataset, and the final accuracy of constitution recognition is 66.79%.

[1]  Shu Liu,et al.  Path Aggregation Network for Instance Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[2]  Guihua Wen,et al.  Natural tongue physique identification using hybrid deep learning methods , 2018, Multimedia Tools and Applications.

[3]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[4]  Jia Liu,et al.  Deep Convolutional Neural Networks for Classifying Body Constitution , 2016, ICANN.

[5]  Rama Chellappa,et al.  Learning Common and Feature-Specific Patterns: A Novel Multiple-Sparse-Representation-Based Tracker , 2018, IEEE Transactions on Image Processing.

[6]  Nima Tajbakhsh,et al.  Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? , 2016, IEEE Transactions on Medical Imaging.

[7]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Jose Miguel Puerta,et al.  Speeding up incremental wrapper feature subset selection with Naive Bayes classifier , 2014, Knowl. Based Syst..

[10]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[11]  Emmanuelle Gouillart,et al.  scikit-image: image processing in Python , 2014, PeerJ.

[12]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[13]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[14]  Wen-Miin Liang,et al.  BCQ+: A Body Constitution Questionnaire to Assess Yang-Xu , 2009, Complementary Medicine Research.

[15]  Shifeng Zhang,et al.  Single-Shot Refinement Neural Network for Object Detection , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[16]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[17]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[18]  Yi-Chang Su,et al.  Measuring body constitution: validation of the Body Constitution Questionnaire (BCQ) in Hong Kong. , 2014, Complementary therapies in medicine.

[19]  Kun-Huang Chen,et al.  Applying particle swarm optimization-based decision tree classifier for cancer classification on gene expression data , 2014, Appl. Soft Comput..

[20]  Borko Furht,et al.  Rethinking Skin Lesion Segmentation in a Convolutional Classifier , 2018, Journal of Digital Imaging.

[21]  Yi-Chang Su,et al.  BCQs: A Body Constitution Questionnaire to assess Stasis in traditional Chinese medicine , 2012 .

[22]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Kaveh Pahlavan,et al.  Enlighten Wearable Physiological Monitoring Systems: On-Body RF Characteristics Based Human Motion Classification Using a Support Vector Machine , 2016, IEEE Transactions on Mobile Computing.

[24]  Yi-Chang Su,et al.  BCQ–: A Body Constitution Questionnaire to Assess Yin-Xu , 2012, Complementary Medicine Research.

[25]  Yi-Chang Su,et al.  BCQ–: A Body Constitution Questionnaire to Assess Yin-Xu , 2012, Complementary Medicine Research.

[26]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[27]  Qing Wang,et al.  Deep Convolutional Neural Networks for Classifying Body Constitution Based on Face Image , 2017, Comput. Math. Methods Medicine.

[28]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[29]  Marleen de Bruijne,et al.  Transfer Learning Improves Supervised Image Segmentation Across Imaging Protocols , 2015, IEEE Trans. Medical Imaging.

[30]  Lijun Jiang,et al.  Tongue image constitution recognition based on Complexity Perception method , 2018, ArXiv.

[31]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Daniel S. Kermany,et al.  Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning , 2018, Cell.

[33]  Ammad Ali,et al.  Face Recognition with Local Binary Patterns , 2012 .

[34]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Wei Jiang,et al.  k-Nearest Neighbor Classification over Semantically Secure Encrypted Relational Data , 2014, IEEE Transactions on Knowledge and Data Engineering.

[36]  Abdulhamit Subasi,et al.  Congestive heart failure detection using random forest classifier , 2016, Comput. Methods Programs Biomed..

[37]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Hei Law,et al.  CornerNet: Detecting Objects as Paired Keypoints , 2018, International Journal of Computer Vision.

[39]  Chuang-Chien Chiu,et al.  Acoustic features for identifying constitutions in traditional Chinese medicine. , 2013, Journal of alternative and complementary medicine.

[40]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[41]  Wang Rui,et al.  Development of Constitution Theory in TCM and Integrated TCM and Western Medicine , 2005 .

[42]  Pong C. Yuen,et al.  Learning Modality-Consistency Feature Templates: A Robust RGB-Infrared Tracking System , 2019, IEEE Transactions on Industrial Electronics.

[43]  Dahua Lin,et al.  PolyNet: A Pursuit of Structural Diversity in Very Deep Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  In-So Kweon,et al.  CBAM: Convolutional Block Attention Module , 2018, ECCV.

[45]  Andrew Y. Ng,et al.  CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning , 2017, ArXiv.

[46]  Rama Chellappa,et al.  Joint Sparse Representation and Robust Feature-Level Fusion for Multi-Cue Visual Tracking , 2015, IEEE Transactions on Image Processing.

[47]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[48]  Jun Dong,et al.  Wrist pulse signals analysis based on Deep Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology.

[49]  Xiaogang Wang,et al.  Residual Attention Network for Image Classification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Yi-Chang Su,et al.  BCQ+: A Body Constitution Questionnaire to Assess Yang-Xu , 2008, Complementary Medicine Research.

[51]  Wang Qi,et al.  Suggestion and analysis on revise of standard of Classification and Determination of Constitution in TCM , 2013 .

[52]  Gang Sun,et al.  Squeeze-and-Excitation Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[53]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[54]  A GámezJosé,et al.  Speeding up incremental wrapper feature subset selection with Naive Bayes classifier , 2014 .

[55]  Hui Li,et al.  Digital mammographic tumor classification using transfer learning from deep convolutional neural networks , 2016, Journal of medical imaging.