Towards content-based patent image retrieval: A framework perspective

In this article, we discuss the potential benefits, the requirements and the challenges involved in patent image retrieval and subsequently, we propose a framework that encompasses advanced image analysis and indexing techniques to address the need for content-based patent image search and retrieval. The proposed framework involves the application of document image pre-processing, image feature and textual metadata extraction in order to support effectively content-based image retrieval in the patent domain. To evaluate the capabilities of our proposal, we implemented a patent image search engine. Results based on a series of interaction modes, comparison with existing systems and a quantitative evaluation of our engine provide evidence that image processing and indexing technologies are currently sufficiently mature to be integrated in real-world patent retrieval applications.

[1]  R. Brunelli,et al.  A Survey on Video Indexing , 1996 .

[2]  Jim Austin,et al.  Trademark image retrieval using multiple features , 1999 .

[3]  Emanuele Pianta,et al.  Integration of Semantic, Metadata and Image Search Engines with a Text Search Engine for Patent Retrieval , 2008, SemSearch.

[4]  Whoi-Yul Kim,et al.  Content-based trademark retrieval system using a visually salient feature , 1998, Image Vis. Comput..

[5]  David Newton Information Retrieval Facility Symposium (IRFS), Vienna, Austria, November 2008 , 2009 .

[6]  Yiannis Kompatsiaris,et al.  A hybrid ontology and visual-based retrieval model for cultural heritage multimedia collections , 2008, Int. J. Metadata Semant. Ontologies.

[7]  Stephen Adams Electronic non-text material in patent applications—some questions for patent offices, applicants and searchers , 2005 .

[8]  Frank Hönes,et al.  Layout extraction of mixed mode documents , 2005, Machine Vision and Applications.

[9]  Xiaogang Wang,et al.  World Wide Web Based Image Search Engine Using Text and Image Content Features , 2003, IS&T/SPIE Electronic Imaging.

[10]  Yiannis Kompatsiaris,et al.  Towards content-oriented patent document processing , 2008 .

[11]  Veena Bansal,et al.  PATSEEK: Content Based Image Retrieval System for Patent Database , 2004, ICEB.

[12]  Jiwu Huang,et al.  Near-Duplicate Image Recognition and Content-based Image Retrieval using Adaptive Hierarchical Geometric Centroids , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[13]  Jane List,et al.  How drawings could enhance retrieval in mechanical and device patent searching , 2007 .

[14]  Michael S. Lew,et al.  Principles of Visual Information Retrieval , 2001, Advances in Pattern Recognition.

[15]  Michael Blackman IPI-ConfEx conference and exposition, Venice-Mestre, Italy, March 2009 , 2009 .

[16]  Jeff Z. Pan,et al.  Resource Description Framework , 2020, Definitions.

[17]  Bernard Mérialdo,et al.  Relational skeletons for retrieval in patent drawings , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[18]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  John P. Eakins Trademark Image Retrieval , 2001, Principles of Visual Information Retrieval.

[20]  R. Brunelli,et al.  A Survey on the Automatic Indexing of Video Data, , 1999, J. Vis. Commun. Image Represent..

[21]  Remco C. Veltkamp,et al.  Practice and challenges in trademark image retrieval , 2007, CIVR '07.

[22]  Nigel Shadbolt,et al.  Resource Description Framework (RDF) , 2009 .

[23]  Raimondo Schettini,et al.  Similarity retrieval of trademark images , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[24]  Babu M. Mehtre,et al.  Content-based retrieval for trademark registration , 1996, Multimedia Tools and Applications.

[25]  Xu Bin,et al.  An Outward-Appearance Patent-Image Retrieval Approach Based on the Contour-Description Matrix , 2007, 2007 Japan-China Joint Workshop on Frontier of Computer Science and Technology (FCST 2007).

[26]  Michael G. Strintzis,et al.  Advanced content-based semantic scene analysis and information retrieval: the SCHEMA project , 2003 .

[27]  Anil K. Jain,et al.  Shape-Based Retrieval: A Case Study With Trademark Image Databases , 1998, Pattern Recognit..