A novel image retrieval algorithm based on transfer learning and fusion features

With proliferation of social media, image has become ubiquitous giving rise to the demand and importance of image semantic analysis and retrieval to access information quickly on social media. However, even with humongous information available, there are certain categories of images which are important for certain applications but are very scarce. Convolutional neural network is an effective method to extract high-level semantic features for image database retrieval. To overcome the problem of over-fitting when the number of training samples in dataset is limited, this paper proposes an image database retrieval algorithm based on the framework of transfer learning and feature fusion. Based on the fine-tuning of the pre-trained Convolutional Neural Network (CNN), the proposed algorithm first extracts the semantic features of the images. Principal Component Analysis (PCA) is then applied for dimension reduction and to reduce the computational complexity. Last, the semantic feature extracted from the CNN is fused with traditional low-level visual feature to improve the retrieval accuracy further. Experimental results demonstrated the effectiveness of the proposed method for image database retrieval.

[1]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[2]  Ronan Sicre,et al.  Particular object retrieval with integral max-pooling of CNN activations , 2015, ICLR.

[3]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Victor S. Lempitsky,et al.  Neural Codes for Image Retrieval , 2014, ECCV.

[5]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[6]  Qi Tian,et al.  Bayes Merging of Multiple Vocabularies for Scalable Image Retrieval , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[9]  M. S. Hitam,et al.  Content-Based Image Retrieval using SIFT for binary and greyscale images , 2013, 2013 IEEE International Conference on Signal and Image Processing Applications.

[10]  G. Yamuna,et al.  Segmentation of natural colour image based on colour-texture features , 2013, 2013 International Conference on Communication and Signal Processing.

[11]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[13]  Harris Drucker,et al.  Learning algorithms for classification: A comparison on handwritten digit recognition , 1995 .

[14]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[15]  Priyadarsan Parida,et al.  2-D Gabor filter based transition region extraction and morphological operation for image segmentation , 2017, Comput. Electr. Eng..

[16]  Jing Dong,et al.  What Is the Best Practice for CNNs Applied to Visual Instance Retrieval? , 2016, ArXiv.

[17]  Liujuan Cao,et al.  Robust latent semantic exploration for image retrieval in social media , 2015, Neurocomputing.

[18]  Yoshua Bengio,et al.  Deep Learning of Representations for Unsupervised and Transfer Learning , 2011, ICML Unsupervised and Transfer Learning.

[19]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Fuping Wang,et al.  Multi-Feature Fusion for Crime Scene Investigation Image Retrieval , 2017, 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[21]  Chengcui Zhang,et al.  Social Media Image Retrieval Using Distilled Convolutional Neural Network for Suspicious e-Crime and Terrorist Account Detection , 2016, 2016 IEEE International Symposium on Multimedia (ISM).

[22]  Andrea Vedaldi,et al.  MatConvNet: Convolutional Neural Networks for MATLAB , 2014, ACM Multimedia.

[23]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[24]  Qi Tian,et al.  Packing and Padding: Coupled Multi-index for Accurate Image Retrieval , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Angel Domingo Sappa,et al.  Fine-Tuning Based Deep Convolutional Networks for Lepidopterous Genus Recognition , 2016, CIARP.

[26]  Jing-Yu Yang,et al.  Content-based image retrieval using computational visual attention model , 2015, Pattern Recognit..

[27]  Seok-Wun Ha,et al.  ROI Based Natural Image Retrieval Using Color and Texture Feature , 2007, Fourth International Conference on Fuzzy Systems and Knowledge Discovery (FSKD 2007).

[28]  Qi Tian,et al.  Coupled Binary Embedding for Large-Scale Image Retrieval , 2014, IEEE Transactions on Image Processing.

[29]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Hiroyuki Kitagawa,et al.  GPU Acceleration of Content-Based Image Retrieval Based on SIFT Descriptors , 2016, 2016 19th International Conference on Network-Based Information Systems (NBiS).

[31]  Xiangde Zhang,et al.  Feature Extraction and Fusion Using Deep Convolutional Neural Networks for Face Detection , 2017 .

[32]  Simon Osindero,et al.  Cross-Dimensional Weighting for Aggregated Deep Convolutional Features , 2015, ECCV Workshops.