Boundary Guidance Hierarchical Network for Real-Time Tongue Segmentation

Automated tongue image segmentation in tongue images is a challenging task for two reasons: 1) there are many pathological details on the tongue surface, which affect the extraction of the boundary; 2) the shapes of the tongues captured from various persons (with different diseases) are quite different. To deal with the challenge, a novel end-to-end Boundary Guidance Hierarchical Network (BGHNet) with a new hybrid loss is proposed in this paper. In the new approach, firstly Context Feature Encoder Module (CFEM) is built upon the bottomup pathway to confront with the shrinkage of the receptive field. Secondly, a novel hierarchical recurrent feature fusion module (HRFFM) is adopt to progressively and hierarchically refine object maps to recover image details by integrating local context information. Finally, the proposed hybrid loss in a four hierarchy-pixel, patch, map and boundary guides the network to effectively segment the tongue regions and accurate tongue boundaries. BGHNet is applied to a set of tongue images. The experimental results suggest that the proposed approach can achieve the latest tongue segmentation performance. And in the meantime, the lightweight network contains only 15.45M parameters and performs only 11.22GFLOPS.

[1]  Gang Yu,et al.  BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation , 2018, ECCV.

[2]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[3]  Maciej Zieba,et al.  Service-Oriented Medical System for Supporting Decisions With Missing and Imbalanced Data , 2014, IEEE Journal of Biomedical and Health Informatics.

[4]  Bob Zhang,et al.  TongueNet: A Precise and Fast Tongue Segmentation System Using U-Net with a Morphological Processing Layer , 2019, Applied Sciences.

[5]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[6]  Marzia Hoque Tania,et al.  Advances in automated tongue diagnosis techniques , 2018, Integrative medicine research.

[7]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[8]  Zhuo Li.  An Improved Snakes Model for Tongue Image Segmentation , 2011 .

[9]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Subhransu Maji,et al.  Semantic contours from inverse detectors , 2011, 2011 International Conference on Computer Vision.

[11]  David Zhang,et al.  Automatic tongue image segmentation based on gradient vector flow and region merging , 2010, Neural Computing and Applications.

[12]  Gang Yu,et al.  Learning a Discriminative Feature Network for Semantic Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[13]  Zuoyong Li,et al.  Tonguenet: Accurate Localization and Segmentation for Tongue Images Using Deep Neural Networks , 2019, IEEE Access.

[14]  Shie Mannor,et al.  A Tutorial on the Cross-Entropy Method , 2005, Ann. Oper. Res..

[15]  Christopher C. Yang,et al.  Intelligent healthcare informatics in big data era , 2015, Artif. Intell. Medicine.

[16]  Shuangge Ma,et al.  Impact of the new health care reform on hospital expenditure in China: A case study from a pilot city , 2016 .

[17]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[21]  Gabriela Csurka,et al.  What is a good evaluation measure for semantic segmentation? , 2013, BMVC.

[22]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Christopher Dainton,et al.  A review of electronic medical record keeping on mobile medical service trips in austere settings , 2017, Int. J. Medical Informatics.

[24]  Zhang Zhishun.  Improvement wavelet transformation in Chinese medicine pip edge examination research , 2012 .

[25]  Yu Wang,et al.  Lednet: A Lightweight Encoder-Decoder Network for Real-Time Semantic Segmentation , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[26]  Fu Zhi-cheng.  Tongue Image Segmentation Based on Snake Model and Radial Edge Detection , 2009 .

[27]  Chao Gao,et al.  BASNet: Boundary-Aware Salient Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Seunghoon Hong,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[29]  David Zhang,et al.  Tongue image analysis for appendicitis diagnosis , 2005, Inf. Sci..

[30]  Zhou Chang-le.  Effective Algorithm on Automatic Segmentation of Body of Tongue , 2007 .

[31]  Yanyun Qu,et al.  Deeptongue: Tongue Segmentation Via Resnet , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[32]  Ping Feng,et al.  Telehealth attitudes and use among medical professionals, medical students and patients in China: A cross-sectional survey , 2017, Int. J. Medical Informatics.

[33]  Ming-Yu Liu,et al.  CASENet: Deep Category-Aware Semantic Edge Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[35]  Wei Liu,et al.  ParseNet: Looking Wider to See Better , 2015, ArXiv.

[36]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[37]  Ralph Deters,et al.  Mobile Medical Data Synchronization on Cloud-Powered Middleware Platform , 2016, IEEE Transactions on Services Computing.

[38]  Hao Chen,et al.  DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Lizhong Zhang,et al.  Application of Image Segmentation Technique in Tongue Diagnosis , 2009, 2009 International Forum on Information Technology and Applications.

[40]  Jianbo Shi,et al.  Semantic Segmentation with Boundary Neural Fields , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Raquel Urtasun,et al.  Understanding the Effective Receptive Field in Deep Convolutional Neural Networks , 2016, NIPS.

[42]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Eugenio Culurciello,et al.  ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation , 2016, ArXiv.

[44]  Evgeny Burnaev,et al.  Boundary Loss for Remote Sensing Imagery Semantic Segmentation , 2019, ISNN.