A Stacked Multi-Granularity Convolution Denoising Auto-Encoder

With the development of big data, artificial intelligence has provided many intelligent solutions to urban life. For instance, an image-based intelligent technology, such as image classification of diseases, is widely used in daily life. However, the image in real life is mostly unlabeled, so the performance of many image-based intelligent models shows limitations. Therefore, how to use a large amount of unlabeled image data to build an efficient and high-quality model for better urban life has been an urgent research topic. In this paper, we propose an unsupervised image feature extraction method that is referred to as a stacked multi-granularity convolution denoising auto-encoder (SMGCDAE). The algorithm is based on a convolutional neural network (CNN), yet it introduces a multi-granularity kernel. This approach resolved issues with image unicity by extracting a diverse category of high-level features. In addition, the denoising auto-encoder ensures stability and improves the classification accuracy by extracting more robust features. The algorithm was assessed using three image benchmark datasets and a series of meningitis images, achieving higher average accuracy than other methods. These results suggest that the algorithm is capable of extracting more discriminative high-level features and thus offers superior performance compared with the existing methodologies.

[1]  Lin Zhao,et al.  A Deep Feature Optimization Fusion Method for Extracting Bearing Degradation Features , 2018, IEEE Access.

[2]  Yasuo Horiuchi,et al.  Reverberant speech recognition based on denoising autoencoder , 2013, INTERSPEECH.

[3]  Yu Zhang,et al.  Unsupervised domain adaptation for robust speech recognition via variational autoencoder-based data augmentation , 2017, 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU).

[4]  Jonathan Tompson,et al.  Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation , 2014, NIPS.

[5]  Andrew L. Maas.  Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[6]  Ling Shao,et al.  Feature Learning for Image Classification Via Multiobjective Genetic Programming , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Tingxi Wen,et al.  Deep Convolution Neural Network and Autoencoders-Based Unsupervised Feature Learning of EEG Signals , 2018, IEEE Access.

[8]  Mo M. Jamshidi,et al.  Feature Fusion for Denoising and Sparse Autoencoders: Application to Neuroimaging Data , 2016, 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA).

[9]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[10]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[11]  H. Bourlard,et al.  Auto-association by multilayer perceptrons and singular value decomposition , 1988, Biological Cybernetics.

[12]  R. Vaillant,et al.  Original approach for the localisation of objects in images , 1994 .

[13]  Dong Wang,et al.  Music removal by convolutional denoising autoencoder in speech recognition , 2015, 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA).

[14]  Domenec Puig,et al.  Recognizing Traffic Signs Using a Practical Deep Neural Network , 2015, ROBOT.

[15]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[16]  Denis Caromel,et al.  Fine Tuning Algorithmic Skeletons , 2007, Euro-Par.

[17]  Jürgen Schmidhuber,et al.  Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction , 2011, ICANN.

[18]  Jacek M. Zurada,et al.  Training neural network classifiers for medical decision making: The effects of imbalanced datasets on classification performance , 2008, Neural Networks.

[19]  Geoffrey E. Hinton,et al.  Replicated Softmax: an Undirected Topic Model , 2009, NIPS.

[20]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[21]  Shagan Sah,et al.  Adaptive hierarchical classification networks , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[22]  Yun Yang,et al.  Time Series Clustering Via RPCL Network Ensemble With Different Representations , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[23]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[24]  Bo Du,et al.  Saliency-Guided Unsupervised Feature Learning for Scene Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[26]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[27]  Ji Feng,et al.  Deep Forest: Towards An Alternative to Deep Neural Networks , 2017, IJCAI.

[28]  Bin Fang,et al.  Scene classification based on single-layer SAE and SVM , 2015, Expert Syst. Appl..

[29]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[30]  이상헌,et al.  Deep Belief Networks , 2010, Encyclopedia of Machine Learning.

[31]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[32]  Christophe Garcia,et al.  Convolutional face finder: a neural architecture for fast and robust face detection , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Thomas G. Dietterich.  Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[34]  Terrence J. Sejnowski,et al.  Unsupervised Learning , 2018, Encyclopedia of GIS.

[35]  Gary King,et al.  Logistic Regression in Rare Events Data , 2001, Political Analysis.

[36]  Yun Yang,et al.  Hybrid Sampling-Based Clustering Ensemble With Global and Local Constitutions , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[37]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[38]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[39]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[40]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[41]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[42]  C. Zhang,et al.  DNN speaker adaptation using parameterised sigmoid and ReLU hidden activation functions , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[43]  Yun Yang,et al.  Temporal Data Clustering via Weighted Clustering Ensemble with Different Representations , 2011, IEEE Transactions on Knowledge and Data Engineering.

[44]  Yong Luo,et al.  Large Margin Multi-Modal Multi-Task Feature Extraction for Image Classification , 2019, IEEE Transactions on Image Processing.

[45]  Joost van de Weijer,et al.  Author Manuscript, Published in "ieee Transactions on Image Processing Edge-based Color Constancy , 2022 .

[46]  Yan Yang,et al.  Classification of human epithelial type 2 cell images using independent component analysis , 2013, 2013 IEEE International Conference on Image Processing.

[47]  Pascal Vincent,et al.  The Difficulty of Training Deep Architectures and the Effect of Unsupervised Pre-Training , 2009, AISTATS.

[48]  Yun Yang,et al.  A novel parallel distance metric-based approach for diversified ranking on large graphs , 2018, Future Gener. Comput. Syst..

[49]  Junping Du,et al.  Noisy Softmax: Improving the Generalization Ability of DCNN via Postponing the Early Softmax Saturation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  B. K. Julsing,et al.  Face Recognition with Local Binary Patterns , 2012 .

[52]  Tony X. Han,et al.  Multiple Instance Learning Convolutional Neural Networks for object recognition , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[53]  Aditya Bhaskara,et al.  Provable Bounds for Learning Some Deep Representations , 2013, ICML.

[54]  Wei Li,et al.  nsemble-based hybrid probabilistic sampling for imbalanced data earning in lung nodule CAD , 2014 .

[55]  Y. X. Zou,et al.  A Robust Acoustic Feature Extraction Approach Based on Stacked Denoising Autoencoder , 2015, 2015 IEEE International Conference on Multimedia Big Data.

[56]  James A. Koziol,et al.  Restricted Boltzmann Machines for Classification of Hepatocellular Carcinoma , 2014 .

[57]  Changqing Shen,et al.  Stacked Sparse Autoencoder-Based Deep Network for Fault Diagnosis of Rotating Machinery , 2017, IEEE Access.

[58]  Piero Baraldi,et al.  Differential evolution-based multi-objective optimization for the definition of a health indicator for fault diagnostics and prognostics , 2018 .

[59]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[60]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[61]  Jamie B. Coble,et al.  A Review of Prognostics and Health Management Applications in Nuclear Power Plants , 2020, International Journal of Prognostics and Health Management.

[62]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.