The Design Patent Images Classification Based on Image Caption Model

Improving the performance of the patented image retrieval system is of great significance in the intellectual property protection. The design patent image has a large amount of data, and how to quickly complete the retrieval is part of the main research issues for the design patent retrieval system. Classification is an effective way to improve the retrieval speed, so some methods of image classification have been proposed before. However, image classification cannot achieve high-level semantic classification. Thus the speed of improvement is very limited. In order to realize the classification effect of high-level semantics, in this paper, we propose a method that uses the image caption model-based to realize the automatic description generation of the design patent image. Experiments show that our method has better classification accuracy and better semantic classification performance than previous image classification methods.

[1]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Jinchang Ren,et al.  Monte Carlo Convex Hull Model for classification of traditional Chinese paintings , 2016, Neurocomputing.

[3]  Heng Tao Shen,et al.  Hierarchical LSTMs with Adaptive Attention for Visual Captioning , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Yijun Yan,et al.  Fusion of block and keypoints based approaches for effective copy-move image forgery detection , 2016, Multidimens. Syst. Signal Process..

[5]  Stephen Marshall,et al.  Cognitive Fusion of Thermal and Visible Imagery for Effective Detection and Tracking of Pedestrians in Videos , 2018, Cognitive Computation.

[6]  Dong Wang,et al.  Effective recognition of MCCs in mammograms using an improved neural classifier , 2011, Eng. Appl. Artif. Intell..

[7]  Wlodek Zadrozny,et al.  Patent retrieval: a literature review , 2017, Knowledge and Information Systems.

[8]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[9]  Xuelong Li,et al.  Unsupervised image saliency detection with Gestalt-laws guided optimization and visual attention based refinement , 2018, Pattern Recognit..

[10]  Xiaosong Zhao,et al.  Image caption model of double LSTM with scene factors , 2019, Image Vis. Comput..

[11]  Ming Xu,et al.  Multi-camera video surveillance for real-time analysis and reconstruction of soccer games , 2010, Machine Vision and Applications.

[12]  Md. Zakir Hossain,et al.  A Comprehensive Survey of Deep Learning for Image Captioning , 2018, ACM Comput. Surv..

[13]  Haitao Huang,et al.  Abstractive text summarization using LSTM-CNN based deep learning , 2018, Multimedia Tools and Applications.

[14]  Jinchang Ren,et al.  Efficient detection of temporally impulsive dirt impairments in archived films , 2007, Signal Process..

[15]  Ji Wan,et al.  Deep Learning for Content-Based Image Retrieval: A Comprehensive Study , 2014, ACM Multimedia.

[16]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[17]  Yiannis Kompatsiaris,et al.  Enhancing Patent Search with Content-Based Image Retrieval , 2014, Professional Search in the Modern World.

[18]  Josh Lerner,et al.  Intellectual Property Rights Protection, Ownership, and Innovation: Evidence from China , 2016 .

[19]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[20]  Richard Socher,et al.  Knowing When to Look: Adaptive Attention via a Visual Sentinel for Image Captioning , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Muhammad Sharif,et al.  Content Based Image Retrieval: Survey , 2012 .

[22]  Peter Young,et al.  Framing Image Description as a Ranking Task: Data, Models and Evaluation Metrics , 2013, J. Artif. Intell. Res..

[23]  Samy Bengio,et al.  Show and tell: A neural image caption generator , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Gabriela Csurka,et al.  Document image classification, with a specific view on applications of patent images , 2016, ArXiv.

[25]  Lei Zhang,et al.  Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[26]  Hermann Ney,et al.  From Feedforward to Recurrent LSTM Neural Networks for Language Modeling , 2015, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[27]  Symeon Papadopoulos,et al.  Towards content-based patent image retrieval: A framework perspective , 2010 .

[28]  Yan Zhou,et al.  Hierarchical Visual Perception and Two-Dimensional Compressive Sensing for Effective Content-Based Color Image Retrieval , 2016, Cognitive Computation.

[29]  Zhenhua Guo,et al.  Patent Image Classification Using Local-Constrained Linear Coding and Spatial Pyramid Matching , 2015, 2015 International Conference on Service Science (ICSS).

[30]  Cao Lu.  Design patent image retrieval system based on semantic classification , 2012 .

[31]  Xiaodong Li,et al.  A Dynamic Neighborhood Learning-Based Gravitational Search Algorithm , 2018, IEEE Transactions on Cybernetics.