A Feature Fusion Method with Guided Training for Classification Tasks

In this paper, a feature fusion method with guiding training (FGT-Net) is constructed to fuse image data and numerical data for some specific recognition tasks which cannot be classified accurately only according to images. The proposed structure is divided into the shared weight network part, the feature fused layer part, and the classification layer part. First, the guided training method is proposed to optimize the training process, the representative images and training images are input into the shared weight network to learn the ability that extracts the image features better, and then the image features and numerical features are fused together in the feature fused layer to input into the classification layer for the classification task. Experiments are carried out to verify the effectiveness of the proposed model. Loss is calculated by the output of both the shared weight network and classification layer. The results of experiments show that the proposed FGT-Net achieves the accuracy of 87.8%, which is 15% higher than the CNN model of ShuffleNetv2 (which can process image data only) and 9.8% higher than the DNN method (which processes structured data only).

[1]  Magudeeswaran Veluchamy,et al.  Fuzzy dissimilarity color histogram equalization for contrast enhancement and color correction , 2020, Appl. Soft Comput..

[2]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[3]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[4]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[5]  Haidong Shao,et al.  An enhancement deep feature fusion method for rotating machinery fault diagnosis , 2017, Knowl. Based Syst..

[6]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Francisco Herrera,et al.  Coral species identification with texture or structure images using a two-level classifier based on Convolutional Neural Networks , 2019, Knowl. Based Syst..

[8]  Stefanos Zafeiriou,et al.  ArcFace: Additive Angular Margin Loss for Deep Face Recognition , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[10]  Xiangyu Zhang,et al.  ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Minu George,et al.  Fuzzy-rough assisted refinement of image processing procedure for mammographic risk assessment , 2020, Appl. Soft Comput..

[12]  M. Cacciola,et al.  Swarm Optimization for Imaging of Corrosion by Impedance Measurements in Eddy Current Test , 2006, 2006 12th Biennial IEEE Conference on Electromagnetic Field Computation.

[13]  Zheng Liu,et al.  Multi-source urban data fusion for property value assessment: A case study in Philadelphia , 2020, Neurocomputing.

[14]  M. M. Ruiz,et al.  A tutorial on ensembles and deep learning fusion with MNIST as guiding thread: A complex heterogeneous fusion scheme reaching 10 digits error , 2020, ArXiv.

[15]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Bin Hu,et al.  Feature-level fusion approaches based on multimodal EEG data for depression recognition , 2020, Inf. Fusion.

[17]  Weihong Deng,et al.  Orthogonality Loss: Learning Discriminative Representations for Face Recognition , 2020 .

[18]  Bernard De Baets,et al.  Deep feature fusion through adaptive discriminative metric learning for scene recognition , 2020, Inf. Fusion.

[19]  Francesco Carlo Morabito,et al.  Adaptive Image Contrast Enhancement by Computing Distances into a 4-Dimensional Fuzzy Unit Hypercube , 2017, IEEE Access.

[20]  Jie Cao,et al.  Dual Cross-Entropy Loss for Small-Sample Fine-Grained Vehicle Classification , 2019, IEEE Transactions on Vehicular Technology.

[21]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Meng Yang,et al.  Large-Margin Softmax Loss for Convolutional Neural Networks , 2016, ICML.

[23]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[24]  Larry S. Davis,et al.  An Analysis of Scale Invariance in Object Detection - SNIP , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[25]  Xianguo Wu,et al.  Multi-classifier information fusion in risk analysis , 2020, Inf. Fusion.

[26]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[27]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[28]  Simon Haykin,et al.  GradientBased Learning Applied to Document Recognition , 2001 .

[29]  Jian Cheng,et al.  Additive Margin Softmax for Face Verification , 2018, IEEE Signal Processing Letters.

[30]  Bhiksha Raj,et al.  SphereFace: Deep Hypersphere Embedding for Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[32]  Marios Savvides,et al.  Ring Loss: Convex Feature Normalization for Face Recognition , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[33]  H. Khotanlou,et al.  Image contrast enhancement using fuzzy clustering with adaptive cluster parameter and sub-histogram equalization , 2017, Digit. Signal Process..

[34]  Xiangyu Zhang,et al.  ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design , 2018, ECCV.

[35]  Tong Liu,et al.  Convolutional Neural Network and Guided Filtering for SAR Image Denoising , 2019, Remote. Sens..

[36]  Jie Zhao,et al.  Deep Spatio-Temporal Representation and Ensemble Classification for Attention Deficit/Hyperactivity Disorder , 2020, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[37]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[38]  S. M. Riazul Islam,et al.  A smart healthcare monitoring system for heart disease prediction based on ensemble deep learning and feature fusion , 2020, Inf. Fusion.

[39]  Quoc V. Le,et al.  Searching for MobileNetV3 , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[40]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[42]  Chen Chen,et al.  Pan-GAN: An unsupervised pan-sharpening method for remote sensing image fusion , 2020, Inf. Fusion.

[43]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[44]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[45]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[46]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[47]  Huimin Zhao,et al.  An Improved Quantum-Inspired Differential Evolution Algorithm for Deep Belief Network , 2020, IEEE Transactions on Instrumentation and Measurement.

[48]  Rogério Schmidt Feris,et al.  A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection , 2016, ECCV.

[49]  Elena Deza,et al.  Encyclopedia of Distances , 2014 .

[50]  Yu Wu,et al.  Automated detection of kidney abnormalities using multi-feature fusion convolutional neural networks , 2020, Knowl. Based Syst..

[51]  Kang Hao Cheong,et al.  Multi-level information fusion to alleviate network congestion , 2020, Inf. Fusion.

[52]  Yujie Li,et al.  Deep Fuzzy Hashing Network for Efficient Image Retrieval , 2021, IEEE Transactions on Fuzzy Systems.

[53]  Jun Wang,et al.  Attributed heterogeneous network fusion via collaborative matrix tri-factorization , 2020, Inf. Fusion.