A Convolutional Autoencoder Topology for Classification in High-Dimensional Noisy Image Datasets

Deep convolutional neural networks have shown remarkable performance in the image classification domain. However, Deep Learning models are vulnerable to noise and redundant information encapsulated into the high-dimensional raw input images, leading to unstable and unreliable predictions. Autoencoders constitute an unsupervised dimensionality reduction technique, proven to filter out noise and redundant information and create robust and stable feature representations. In this work, in order to resolve the problem of DL models’ vulnerability, we propose a convolutional autoencoder topological model for compressing and filtering out noise and redundant information from initial high dimensionality input images and then feeding this compressed output into convolutional neural networks. Our results reveal the efficiency of the proposed approach, leading to a significant performance improvement compared to Deep Learning models trained with the initial raw images.

[1]  En Zhu,et al.  Deep Clustering with Convolutional Autoencoders , 2017, ICONIP.

[2]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  David M. W. Powers,et al.  Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation , 2011, ArXiv.

[4]  Kun Zhang,et al.  A Causal View on Robustness of Neural Networks , 2020, NeurIPS.

[5]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[6]  Ioannis E. Livieris,et al.  An Advanced Deep Learning Model for Short-Term Forecasting U.S. Natural Gas Price and Movement , 2020, AIAI Workshops.

[7]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[8]  Qian Du,et al.  Unsupervised Spatial–Spectral Feature Learning by 3D Convolutional Autoencoder for Hyperspectral Classification , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[9]  J. L. Hodges,et al.  Rank Methods for Combination of Independent Experiments in Analysis of Variance , 1962 .

[10]  Ioannis E. Livieris,et al.  A Grey-Box Ensemble Model Exploiting Black-Box Accuracy and White-Box Intrinsic Interpretability , 2020, Algorithms.

[11]  Eero P. Simoncelli,et al.  Statistical Modeling of Images with Fields of Gaussian Scale Mixtures , 2006, NIPS.

[12]  Ivan Nunes da Silva,et al.  Artificial Neural Network Architectures and Training Processes , 2017 .

[13]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Holger H. Hoos,et al.  A survey on semi-supervised learning , 2019, Machine Learning.

[15]  S. Z. Gürbüz,et al.  Deep convolutional autoencoder for radar-based classification of similar aided and unaided human activities , 2018, IEEE Transactions on Aerospace and Electronic Systems.

[16]  Margret Keuper,et al.  Unmasking DeepFakes with simple Features , 2019, ArXiv.

[17]  David I. McLean,et al.  Detection and Analysis of Irregular Streaks in Dermoscopic Images of Skin Lesions , 2013, IEEE Transactions on Medical Imaging.

[18]  Rainer Stiefelhagen,et al.  Taming the Cross Entropy Loss , 2018, GCPR.

[19]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[20]  Ken D. Sauer,et al.  Gaussian mixture Markov random field for image denoising and reconstruction , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[21]  Mohsen Guizani,et al.  Deep Feature Learning for Medical Image Analysis with Convolutional Autoencoder Neural Network , 2017, IEEE Transactions on Big Data.

[22]  Yuhui Zheng,et al.  Recent Progress on Generative Adversarial Networks (GANs): A Survey , 2019, IEEE Access.

[23]  Ognjen Arandjelovic,et al.  Whole Slide Pathology Image Patch Based Deep Classification: An Investigation of the Effects of the Latent Autoencoder Representation and the Loss Function Form , 2021, 2021 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI).

[24]  Ce Liu,et al.  Deep Convolutional Neural Network for Image Deconvolution , 2014, NIPS.

[25]  H. Sebastian Seung,et al.  Natural Image Denoising with Convolutional Networks , 2008, NIPS.

[26]  Hao Li,et al.  Protecting World Leaders Against Deep Fakes , 2019, CVPR Workshops.

[27]  Hitoshi Iyatomi,et al.  Significant Dimension Reduction of 3D Brain MRI using 3D Convolutional Autoencoders , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[28]  Panayiotis E. Pintelas,et al.  Special Issue on Ensemble Learning and Applications , 2020, Algorithms.

[29]  Zhang Yi,et al.  Learning a good representation with unsymmetrical auto-encoder , 2015, Neural Computing and Applications.

[30]  H. Finner On a Monotonicity Problem in Step-Down Multiple Test Procedures , 1993 .

[31]  Panayiotis E. Pintelas,et al.  An Autoencoder Convolutional Neural Network Framework for Sarcopenia Detection Based on Multi-frame Ultrasound Image Slices , 2021, AIAI.

[32]  Ling Shao,et al.  Transfer Learning for Visual Categorization: A Survey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[33]  Ioannis E. Livieris,et al.  An Advanced CNN-LSTM Model for Cryptocurrency Forecasting , 2021, Electronics.

[34]  Ashutosh Kumar Singh,et al.  Machine Learning for High-Throughput Stress Phenotyping in Plants. , 2016, Trends in plant science.

[35]  Ademola E. Ilesanmi,et al.  Methods for image denoising using convolutional neural network: a review , 2021, Complex & Intelligent Systems.

[36]  Lu Sheng,et al.  Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues , 2020, ECCV.

[37]  Adrian Barbu,et al.  Learning real-time MRF inference for image denoising , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Yong Xu,et al.  Deep Learning for Image Denoising: A Survey , 2018, ICGEC.

[39]  Ioannis E. Livieris,et al.  On ensemble techniques of weight-constrained neural networks , 2020, Evolving Systems.

[40]  Sotiris Kotsiantis,et al.  A novel explainable image classification framework: case study on skin cancer and plant disease prediction , 2021, Neural Computing and Applications.

[41]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[42]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Daniel L. Marino,et al.  ResNet Autoencoders for Unsupervised Feature Learning From High-Dimensional Data: Deep Models Resistant to Performance Degradation , 2021, IEEE Access.

[44]  Cha Zhang,et al.  Ensemble Machine Learning: Methods and Applications , 2012 .

[45]  Baining Guo,et al.  Face X-Ray for More General Face Forgery Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Edward H. Adelson,et al.  Learning Gaussian Conditional Random Fields for Low-Level Vision , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  William T. Freeman,et al.  What makes a good model of natural images? , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Xindong Wu,et al.  Object Detection With Deep Learning: A Review , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[49]  Gavin Brown,et al.  Ensemble Learning , 2010, Encyclopedia of Machine Learning and Data Mining.

[50]  Peijun Du,et al.  Novel segmented stacked autoencoder for effective dimensionality reduction and feature extraction in hyperspectral imaging , 2016, Neurocomputing.

[51]  Ioannis E. Livieris,et al.  Explainable Machine Learning Framework for Image Classification Problems: Case Study on Glioma Cancer Prediction , 2020, J. Imaging.

[52]  Andreas Rössler,et al.  FaceForensics++: Learning to Detect Manipulated Facial Images , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[53]  Dariusz Kucharski,et al.  Semi-Supervised Nests of Melanocytes Segmentation Method Using Convolutional Autoencoders , 2020, Sensors.