Hybrid descriptor for placental maturity grading

Placental maturity grading (PMG) is quite essential to assess fetal growth and maternal health. To this date, PMG has mostly relied on the subjective judgment of the clinician, which is time-consuming and may cause wrong estimation due to redundancy and repeatability of the process. To tackle it, we propose an automatic method to stage placental maturity via deep hybrid descriptors based on B-mode ultrasound (BUS) and color Doppler energy (CDE) images. Specifically, convolutional descriptors extracted from multiple deep convolutional neural networks (DCNNs) and hand-crafted features are integrated to get the hybrid descriptors for grading performance boosting. First, different models with various feature layers are combined to obtain hybrid descriptors from images. Second, the transfer learning strategy is also utilized to enhance the grading performance via the deeply represented features. Third, extracted descriptors are encoded by Fisher vector (FV). Finally, we use support vector machine (SVM) as the classifier to grade placental maturity. The experimental results demonstrate that our proposed method could achieve good performance in PMG.

[1]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[2]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[3]  safa el kamel Imaging of the placenta , 2013 .

[4]  Pheng-Ann Heng,et al.  Ultrasound Standard Plane Detection Using a Composite Neural Network Framework , 2017, IEEE Transactions on Cybernetics.

[5]  Brian C. Lovell,et al.  Fisher tensors for classifying human epithelial cells , 2014, Pattern Recognit..

[6]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[7]  Qi Tian,et al.  Good Practice in CNN Feature Transfer , 2016, ArXiv.

[8]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[9]  Douglas A. Reynolds,et al.  Gaussian Mixture Models , 2018, Encyclopedia of Biometrics.

[10]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[11]  Rui Tato Marinho,et al.  Classification and Staging of Chronic Liver Disease From Multimodal Data , 2013, IEEE Transactions on Biomedical Engineering.

[12]  Jana Kosecka,et al.  Deep Convolutional Features for Image Based Retrieval and Scene Categorization , 2015, ArXiv.

[13]  Xiaofeng Zhu,et al.  Unsupervised feature selection by self-paced learning regularization , 2020, Pattern Recognit. Lett..

[14]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[15]  Larry S. Davis,et al.  Exploiting local features from deep networks for image retrieval , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[16]  Svetlana Lazebnik,et al.  Multi-scale Orderless Pooling of Deep Convolutional Activation Features , 2014, ECCV.

[17]  G. Zombori,et al.  Novel placental ultrasound assessment: potential role in pre-gestational diabetic pregnancy. , 2014, Placenta.

[18]  Norman D. Black,et al.  Feature selection for the characterization of ultrasonic images of the placenta using texture classification , 2004, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[19]  D. Charnock-Jones,et al.  Regulation of vascular growth and function in the human placenta. , 2009, Reproduction.

[20]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Bo Peng,et al.  Cascaded Multi-Column RVFL+ Classifier for Single-Modal Neuroimaging-Based Diagnosis of Parkinson's Disease , 2019, IEEE Transactions on Biomedical Engineering.

[22]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Christine O Menias,et al.  Imaging of the placenta: a multimodality pictorial review. , 2009, Radiographics : a review publication of the Radiological Society of North America, Inc.

[24]  Dong Ni,et al.  Automatic Recognition of Fetal Facial Standard Plane in Ultrasound Image via Fisher Vector , 2015, PloS one.

[25]  R J Sokol,et al.  Detection of intrauterine growth retardation: a new use for sonographic placental grading. , 1983, American journal of obstetrics and gynecology.

[26]  Yuan Yao,et al.  Placental maturity evaluation via feature fusion and discriminative learning , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[27]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[28]  Dong Ni,et al.  Automatic staging of placental maturity based on dense descriptor. , 2014, Bio-medical materials and engineering.

[29]  Xuelong Li,et al.  Graph-based learning for segmentation of 3D ultrasound images , 2015, Neurocomputing.

[30]  Yuan Yao,et al.  Automatic grading of placental maturity based on LIOP and fisher vector , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[31]  Xudong Jiang,et al.  Multi-modal and multi-layout discriminative learning for placental maturity staging , 2017, Pattern Recognit..

[32]  R L Berkowitz,et al.  The ultrasonic changes in the maturing placenta and their relation to fetal pulmonic maturity. , 1979, American journal of obstetrics and gynecology.

[33]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[34]  Hao Chen,et al.  Standard Plane Localization in Fetal Ultrasound via Domain Transferred Deep Neural Networks , 2015, IEEE Journal of Biomedical and Health Informatics.

[35]  V. Feldstein,et al.  Ultrasound of the Placenta and Umbilical Cord: A Review , 2011, Ultrasound quarterly.

[36]  J M Rubin,et al.  Power Doppler sonography. , 1996, Radiology.

[37]  Qinghua Huang,et al.  Correspondence - 3-D ultrasonic strain imaging based on a linear scanning system , 2015, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[38]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[39]  Xuelong Li,et al.  Bezier Interpolation for 3-D Freehand Ultrasound , 2015, IEEE Transactions on Human-Machine Systems.

[40]  Cordelia Schmid,et al.  Aggregating Local Image Descriptors into Compact Codes , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  D. Shen,et al.  Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans , 2016, Scientific Reports.

[42]  Yilong Yin,et al.  Distribution-Oriented Aesthetics Assessment With Semantic-Aware Hybrid Network , 2019, IEEE Transactions on Multimedia.

[43]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Ee-Leng Tan,et al.  Automatic placental maturity grading via hybrid learning , 2017, Neurocomputing.

[45]  Thomas Mensink,et al.  Image Classification with the Fisher Vector: Theory and Practice , 2013, International Journal of Computer Vision.

[46]  S. Guerriero,et al.  Clinical applications of colour Doppler energy imaging in the female reproductive tract and pregnancy. , 1999, Human reproduction update.

[47]  Dev Maulik,et al.  Computer analysis of three-dimensional power angiography images of foetal cerebral, lung and placental circulation in normal and high-risk pregnancy. , 2005, Ultrasound in medicine & biology.

[48]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[49]  T. Todros,et al.  Is three‐dimensional power Doppler ultrasound useful in the assessment of placental perfusion in normal and growth‐restricted pregnancies? , 2008, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[50]  Lei Wang,et al.  Encoding High Dimensional Local Features by Sparse Coding Based Fisher Vectors , 2014, NIPS.

[51]  Herong Zheng,et al.  Application of Multi-Classification Support Vector Machine in the B-Placenta Image Classification , 2009, 2009 International Conference on Computational Intelligence and Software Engineering.

[52]  Trevor Darrell,et al.  Do Convnets Learn Correspondence? , 2014, NIPS.

[53]  P Suetens,et al.  Regional strain and strain rate measurements by cardiac ultrasound: principles, implementation and limitations. , 2000, European journal of echocardiography : the journal of the Working Group on Echocardiography of the European Society of Cardiology.

[54]  Subhransu Maji,et al.  Deep filter banks for texture recognition and segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Jose Villar,et al.  The preterm birth syndrome: issues to consider in creating a classification system. , 2012, American journal of obstetrics and gynecology.