Content-Based Image Retrieval Based on ROI Detection and Relevance Feedback

Content-based image retrieval is an important research topic in computer vision. We present a new method that combines region of interest (ROI) detection and relevance feedback. The ROI based approach is more accurate in describing the image content than using global features, and the relevance feedback makes the system to be adaptive to subjective human perception. The feedback information is utilized to discover the subjective ROI perception of a particular user, and it is further employed to recompute the features associated with ROIs with the updated personalized ROI preference. A fast computation technique is proposed to avoid repeating the ROI detection for images in the database. It directly estimates the features of the ROIs, which makes the query process fast and efficient. For illustration of the overall approach, we use the color saliency and wavelet feature saliency to determine the ROIs. Normalized projections are selected to represent the shape features associated with the ROIs. Experimental results show that the proposed system has better performance than the global features based approaches and region based techniques without feedback.

[1]  Ferran Marqués,et al.  Region-based representations of image and video: segmentation tools for multimedia services , 1999, IEEE Trans. Circuits Syst. Video Technol..

[2]  R. Ramakrishnan,et al.  An Optimizer for Heterogeneous Systems with NonStandard Data and Search Capabilities , 1996 .

[3]  David J. Marchette,et al.  The advanced distributed region of interest tool , 1998, Pattern Recognit..

[4]  Mehmet Celenk,et al.  Higher-order spectra (HOS) invariants for shape recognition , 2001, Pattern Recognit..

[5]  C.-C. Jay Kuo,et al.  Texture analysis and classification with tree-structured wavelet transform , 1993, IEEE Trans. Image Process..

[6]  Qi Tian,et al.  Combine user defined region-of-interest and spatial layout for image retrieval , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[7]  Xiaobo Li,et al.  Region-Based Image Retrieval Using Multiple-Features , 2002, VISUAL.

[8]  Borko Furht,et al.  Video and Image Processing in Multimedia Systems , 1995 .

[9]  Remco C. Veltkamp,et al.  A Survey of Content-Based Image Retrieval Systems , 2002 .

[10]  Eric J. Pauwels,et al.  Finding Salient Regions in Images: Nonparametric Clustering for Image Segmentation and Grouping , 1999, Comput. Vis. Image Underst..

[11]  Thomas S. Huang,et al.  Relevance feedback: a power tool for interactive content-based image retrieval , 1998, IEEE Trans. Circuits Syst. Video Technol..

[12]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[14]  B. S. Manjunath,et al.  NeTra: A toolbox for navigating large image databases , 1997, Proceedings of International Conference on Image Processing.

[15]  Martin D. Levine,et al.  Finding a small number of regions in an image using low-level features , 2002, Pattern Recognit..

[16]  K. Wakimoto,et al.  Efficient and Effective Querying by Image Content , 1994 .

[17]  Alex Pentland,et al.  Photobook: Content-based manipulation of image databases , 1996, International Journal of Computer Vision.

[18]  Wei-Ying Ma,et al.  Information embedding based on user's relevance feedback for image retrieval , 1999, Optics East.

[19]  Thomas S. Huang,et al.  Extending image retrieval with group-oriented interface , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[20]  Linda G. Shapiro,et al.  Computer and Robot Vision , 1991 .

[21]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Jorge Herbert de Lira,et al.  Two-Dimensional Signal and Image Processing , 1989 .

[23]  Qiang Yang,et al.  A unified framework for semantics and feature based relevance feedback in image retrieval systems , 2000, ACM Multimedia.

[24]  Thomas S. Huang,et al.  A novel relevance feedback technique in image retrieval , 1999, MULTIMEDIA '99.

[25]  Vito Di Gesù,et al.  Local operators to detect regions of interest , 1997, Pattern Recognit. Lett..

[26]  Thomas S. Huang,et al.  Comparing discriminating transformations and SVM for learning during multimedia retrieval , 2001, MULTIMEDIA '01.

[27]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Hidetoshi Miike,et al.  Determining optical flow from sequential images , 1988, Systems and Computers in Japan.

[29]  Ilaria Bartolini,et al.  Windsurf: region-based image retrieval using wavelets , 1999, Proceedings. Tenth International Workshop on Database and Expert Systems Applications. DEXA 99.

[30]  Henning Biermann,et al.  Regions-of-Interest and Spatial Layout for Content-Based Image Retrieval , 2001, Multimedia Tools and Applications.

[31]  Kien A. Hua,et al.  Image Retrieval Based on Regions of Interest , 2003, IEEE Trans. Knowl. Data Eng..

[32]  Qiang Zhou,et al.  Adaptive shape transform for color image querying , 2003, IS&T/SPIE Electronic Imaging.

[33]  Bo Zhang,et al.  Region-based relevance feedback in image retrieval , 2002, 2002 IEEE International Symposium on Circuits and Systems. Proceedings (Cat. No.02CH37353).