Joint Triplet Autoencoder for histopathological colon cancer nuclei retrieval

Deep learning has shown a great improvement in the performance of visual tasks. Image retrieval is the task of extracting the visually similar images from a database for a query image. The feature matching is performed to rank the images. Various handdesigned features have been derived in past to represent the images. Nowadays, the power of deep learning is being utilized for automatic feature learning from data in the field of biomedical image analysis. Autoencoder and Siamese networks are two deep learning models to learn the latent space (i.e., features or embedding). Autoencoder works based on the reconstruction of the image from latent space. Siamese network utilizes the triplets to learn the intra-class similarity and inter-class dissimilarity. Moreover, Autoencoder is unsupervised, whereas Siamese network is supervised. We propose a Joint Triplet Autoencoder Network (JTANet) by facilitating the triplet learning in autoencoder framework. A joint supervised learning for Siamese network and unsupervised learning for Autoencoder is performed. Moreover, the Encoder network of Autoencoder is shared with Siamese network and referred as the Siamcoder network. The features are extracted by using the trained Siamcoder network for retrieval purpose. The experiments are performed over Histopathological Routine Colon Cancer dataset. We have observed the promising performance using the proposed JTANet model against the Autoencoder and Siamese models for colon cancer nuclei retrieval in histopathological images.

[1]  Subrahmanyam Murala,et al.  Local ternary co-occurrence patterns: A new feature descriptor for MRI and CT image retrieval , 2013, Neurocomputing.

[2]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Jijun Tang,et al.  DeepAVP: A Dual-Channel Deep Neural Network for Identifying Variable-Length Antiviral Peptides , 2020, IEEE Journal of Biomedical and Health Informatics.

[4]  Shiv Ram Dubey,et al.  Local Diagonal Extrema Pattern: A New and Efficient Feature Descriptor for CT Image Retrieval , 2015, IEEE Signal Processing Letters.

[5]  Bidyut Baran Chaudhuri,et al.  diffGrad: An Optimization Method for Convolutional Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[7]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Nasir M. Rajpoot,et al.  Locality Sensitive Deep Learning for Detection and Classification of Nuclei in Routine Colon Cancer Histology Images , 2016, IEEE Trans. Medical Imaging.

[10]  Hayit Greenspan,et al.  An Adversarial Learning Approach to Medical Image Synthesis for Lesion Detection , 2018, IEEE Journal of Biomedical and Health Informatics.

[11]  Song Bai,et al.  Deep learning representation using autoencoder for 3D shape retrieval , 2014, Proceedings 2014 IEEE International Conference on Security, Pattern Analysis, and Cybernetics (SPAC).

[12]  Shiguang Shan,et al.  Deep Supervised Hashing for Fast Image Retrieval , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Naima Iltaf,et al.  Content-based histopathological image retrieval using multi-scale and multichannel decoder based LTP , 2019, Biomed. Signal Process. Control..

[14]  Geoffrey E. Hinton,et al.  Using very deep autoencoders for content-based image retrieval , 2011, ESANN.

[15]  Shiv Ram Dubey,et al.  A Performance Evaluation of Convolutional Neural Networks for Face Anti Spoofing , 2018, 2019 International Joint Conference on Neural Networks (IJCNN).

[16]  Sunil Kumar,et al.  A new approach for effective retrieval and indexing of medical images , 2019, Biomed. Signal Process. Control..

[17]  Bidyut Baran Chaudhuri,et al.  Local bit-plane decoded convolutional neural network features for biomedical image retrieval , 2019, Neural Computing and Applications.

[18]  Shiv Ram Dubey,et al.  CDGAN: Cyclic Discriminative Generative Adversarial Networks for Image-to-Image Transformation , 2020, J. Vis. Commun. Image Represent..

[19]  Ankan Ghosh Dastider,et al.  An integrated autoencoder-based hybrid CNN-LSTM model for COVID-19 severity prediction from lung ultrasound , 2021, Computers in Biology and Medicine.

[20]  Shiv Ram Dubey,et al.  Local Bit-Plane Decoded Pattern: A Novel Feature Descriptor for Biomedical Image Retrieval , 2016, IEEE Journal of Biomedical and Health Informatics.

[21]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Buddha Singh,et al.  AE-LGBM: Sequence-Based Novel Approach To Detect Interacting Protein Pairs via Ensemble of Autoencoder and LightGBM , 2020, bioRxiv.

[23]  Yuxin Peng,et al.  SSDH: Semi-Supervised Deep Hashing for Large Scale Image Retrieval , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Jun Xu,et al.  Fusing Heterogeneous Features From Stacked Sparse Autoencoder for Histopathological Image Analysis , 2016, IEEE Journal of Biomedical and Health Informatics.

[25]  Tao Mei,et al.  Deep Semantic-Preserving and Ranking-Based Hashing for Image Retrieval , 2016, IJCAI.

[26]  Jianmin Wang,et al.  Deep Quantization Network for Efficient Image Retrieval , 2016, AAAI.

[27]  Shiv Ram Dubey,et al.  Multichannel Decoded Local Binary Patterns for Content-Based Image Retrieval , 2016, IEEE Transactions on Image Processing.

[28]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Subrahmanyam Murala,et al.  Local Tetra Patterns: A New Feature Descriptor for Content-Based Image Retrieval , 2012, IEEE Transactions on Image Processing.

[30]  Wu-Jun Li,et al.  Asymmetric Deep Supervised Hashing , 2017, AAAI.

[31]  Tieniu Tan,et al.  Deep Supervised Discrete Hashing , 2017, NIPS.

[32]  Mehul Motani,et al.  Optimizing Autoencoders for Learning Deep Representations From Health Data , 2019, IEEE Journal of Biomedical and Health Informatics.

[33]  Yi Shi,et al.  Deep Supervised Hashing with Triplet Labels , 2016, ACCV.

[34]  Shiv Ram Dubey,et al.  Local Wavelet Pattern: A New Feature Descriptor for Image Retrieval in Medical CT Databases , 2015, IEEE Transactions on Image Processing.

[35]  Xiaonan Luo,et al.  Automated and precise recognition of human zygote cytoplasm: A robust image-segmentation system based on a convolutional neural network , 2021, Biomed. Signal Process. Control..

[36]  Snehasis Mukherjee,et al.  Spontaneous Facial Micro-Expression Recognition using 3D Spatiotemporal Convolutional Neural Networks , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[37]  Shiv Ram Dubey,et al.  Average biased ReLU based CNN descriptor for improved face retrieval , 2018, Multimedia Tools and Applications.

[38]  Subrahmanyam Murala,et al.  Local Mesh Patterns Versus Local Binary Patterns: Biomedical Image Indexing and Retrieval , 2014, IEEE Journal of Biomedical and Health Informatics.

[39]  Walid Barhoumi,et al.  Dynamic distance learning for joint assessment of visual and semantic similarities within the framework of medical image retrieval , 2020, Comput. Biol. Medicine.

[40]  Shiv Ram Dubey,et al.  Hard-Mining Loss based Convolutional Neural Network for Face Recognition , 2019, ArXiv.

[41]  Yogesh Kumar,et al.  An efficient and robust approach for biomedical image retrieval using Zernike moments , 2018, Biomed. Signal Process. Control..

[42]  Shiv Ram Dubey,et al.  Rotation and Illumination Invariant Interleaved Intensity Order-Based Local Descriptor , 2014, IEEE Transactions on Image Processing.

[43]  Shiqi Wang,et al.  Learning Generalized Deep Feature Representation for Face Anti-Spoofing , 2018, IEEE Transactions on Information Forensics and Security.

[44]  Jing Liu,et al.  Deep Incremental Hashing Network for Efficient Image Retrieval , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Philip S. Yu,et al.  HashNet: Deep Learning to Hash by Continuation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[46]  Linda K. Olson,et al.  Breast mass segmentation in ultrasound with selective kernel U-Net convolutional neural network , 2020, Biomed. Signal Process. Control..

[47]  Snehasis Mukherjee,et al.  RCCNet: An Efficient Convolutional Neural Network for Histological Routine Colon Cancer Nuclei Classification , 2018, 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV).

[48]  Shihong Du,et al.  Spectral–Spatial Feature Extraction for Hyperspectral Image Classification: A Dimension Reduction and Deep Learning Approach , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[49]  Min-Ying Su,et al.  A multi-resolution approach for spinal metastasis detection using deep Siamese neural networks , 2017, Comput. Biol. Medicine.

[50]  Bidyut Baran Chaudhuri,et al.  HybridSN: Exploring 3-D–2-D CNN Feature Hierarchy for Hyperspectral Image Classification , 2019, IEEE Geoscience and Remote Sensing Letters.

[51]  Zhang Xiong,et al.  3D object retrieval with stacked local convolutional autoencoder , 2015, Signal Process..

[52]  Jie Yang,et al.  Densely-Connected Multi-Magnification Hashing for Histopathological Image Retrieval , 2019, IEEE Journal of Biomedical and Health Informatics.

[53]  Diego Castillo-Barnes,et al.  Studying the Manifold Structure of Alzheimer's Disease: A Deep Learning Approach Using Convolutional Autoencoders , 2020, IEEE Journal of Biomedical and Health Informatics.

[54]  Shiv Ram Dubey,et al.  CSGAN: Cyclic-Synthesized Generative Adversarial Networks for Image-to-Image Transformation , 2019, Expert Syst. Appl..

[55]  Matti Pietikäinen,et al.  Computer Vision Using Local Binary Patterns , 2011, Computational Imaging and Vision.

[56]  Saban Öztürk,et al.  Class-driven content-based medical image retrieval using hash codes of deep features , 2021, Biomed. Signal Process. Control..

[57]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[58]  Quoc V. Le A Tutorial on Deep Learning Part 2: Autoencoders, Convolutional Neural Networks and Recurrent Neural Networks , 2015 .

[59]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[60]  Guoying Zhao,et al.  Selective deep features for micro-expression recognition , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[61]  Ronald M. Summers,et al.  TieNet: Text-Image Embedding Network for Common Thorax Disease Classification and Reporting in Chest X-Rays , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[62]  P M Ameer,et al.  Retrieval of brain MRI with tumor using contrastive loss based similarity on GoogLeNet encodings , 2020, Comput. Biol. Medicine.

[63]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[64]  Ying Liu,et al.  A survey of content-based image retrieval with high-level semantics , 2007, Pattern Recognit..