Visual Image Retrieval by Elastic Matching of User Sketches

Effective image retrieval by content from database requires that visual image properties are used instead of textual labels to properly index and recover pictorial data. Retrieval by shape similarity, given a user-sketched template is particularly challenging, owing to the difficulty to derive a similarity measure that closely conforms to the common perception of similarity by humans. In this paper, we present a technique which is based on elastic matching of sketched templates over the shapes in the images to evaluate similarity ranks. The degree of matching achieved and the elastic deformation energy spent by the sketch to achieve such a match are used to derive a measure of similarity between the sketch and the images in the database and to rank images to be displayed. The elastic matching is integrated with arrangements to provide scale invariance and take into account spatial relationships between objects in multi-object queries. Examples from a prototype system are expounded with considerations about the effectiveness of the approach and comparative performance analysis.

[1]  A. N. Tikhonov,et al.  REGULARIZATION OF INCORRECTLY POSED PROBLEMS , 1963 .

[2]  P. Laurent,et al.  A general method for the construction of interpolating or smoothing spline-functions , 1968 .

[3]  Shi-Kuo Chang,et al.  Iconic Indexing by 2-D Strings , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Richard Durbin,et al.  An analogue approach to the travelling salesman problem using an elastic net method , 1987, Nature.

[5]  Shi-Kuo Chang,et al.  An Intelligent Image Database System , 1988, IEEE Trans. Software Eng..

[6]  Suh-Yin Lee,et al.  Similarity retrieval of iconic image database , 1989, Pattern Recognit..

[7]  William I. Grosky,et al.  Index-based object recognition in pictorial data management , 1990, Comput. Vis. Graph. Image Process..

[8]  Raimondo Schettini,et al.  Indexing and Fuzzy Logic-Based Retrieval of Color Images , 1991, Visual Database Systems.

[9]  Alan L. Yuille,et al.  Particle tracking by deformable templates , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[10]  SUH-YIN LEE,et al.  Spatial reasoning and similarity retrieval of images using 2D C-string knowledge representation , 1992, Pattern Recognit..

[11]  Erland Jungert.  The Observer's Point of View: An Extension of Symbolic Projections , 1992, Spatio-Temporal Reasoning.

[12]  Suh-Yin Lee,et al.  Signature file as a spatial filter for iconic image database , 1992, J. Vis. Lang. Comput..

[13]  Toshikazu Kato,et al.  Query by Visual Example - Content based Image Retrieval , 1992, EDBT.

[14]  M. Ohlsson.  Extensions and explorations of the elastic arms algorithm , 1993 .

[15]  Alberto Del Bimbo,et al.  A Three-Dimensional Iconic Environment for Image Database Querying , 1993, IEEE Trans. Software Eng..

[16]  Christos Faloutsos,et al.  QBIC project: querying images by content, using color, texture, and shape , 1993, Electronic Imaging.

[17]  Alberto Del Bimbo,et al.  A Spatial Logic for Symbolic Description of Image Contents , 1994, J. Vis. Lang. Comput..

[18]  K. Wakimoto,et al.  Efficient and Effective Querying by Image Content , 1994 .

[19]  Alberto Del Bimbo,et al.  Visual image retrieval by elastic deformation of object sketches , 1994, Proceedings of 1994 IEEE Symposium on Visual Languages.

[20]  Fang Liu,et al.  A new Wold ordering for image similarity , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[21]  Rajiv Mehrotra,et al.  Similar-Shape Retrieval in Shape Data Management , 1995, Computer.

[22]  David Salesin,et al.  Fast multiresolution image querying , 1995, SIGGRAPH.

[23]  Rosalind W. Picard.  A Society of Models for Video and Image Libraries , 1996, IBM Syst. J..