Multimedia database retrieval using hand-drawn sketches

We propose an integrated approach to color and shape based retrieval which enables the user to search a color image database intuitively by presenting simple sketches. Each of the images is represented by a Hidden Markov Model (HMM) which has been concatenated with modified filler models in order to obtain scale and rotation invariance properties. These properties are particularly important when using query by sketch due to the skew which occurs naturally in human handwriting and in drawings. The use of streams (sets of features that are assumed to be statistically independent) within the HMM framework allows the integration of shape and color derived features into a single model, thereby allowing one to control the influence of the different streams via stream weights. The approach has been evaluated on a color image database containing 120 different isolated objects with arbitrary orientation and showed good retrieval results with several users. Furthermore, the use of HMMs allows efficient pruning and thus a fast retrieval even with large databases.

[1]  Vishwa Gupta,et al.  Integration of acoustic information in a large vocabulary word recognizer , 1987, ICASSP '87. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[2]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[3]  Toshikazu Kato,et al.  Query by Visual Example - Content based Image Retrieval , 1992, EDBT.

[4]  Jerome R. Bellegarda,et al.  On-line handwriting recognition using continuous parameter hidden Markov models , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[5]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[6]  Alberto Del Bimbo,et al.  Visual Image Retrieval by Elastic Matching of User Sketches , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Shi-Nine Yang,et al.  Color image retrieval based on hidden Markov models , 1997, IEEE Trans. Image Process..

[8]  B. Kimia,et al.  Symmetry-based indexing of image databases , 1998, Proceedings. IEEE Workshop on Content-Based Access of Image and Video Libraries (Cat. No.98EX173).

[9]  G. Rigoll,et al.  Image database retrieval of rotated objects by user sketch , 1998, Proceedings. IEEE Workshop on Content-Based Access of Image and Video Libraries (Cat. No.98EX173).

[10]  Benjamin B. Kimia,et al.  Symmetry-Based Indexing of Image Databases , 1998, J. Vis. Commun. Image Represent..