Object and concept recognition for content-based image retrieval

The problem of recognizing classes of objects in images is important for annotation and indexing of image and video databases. Users of commercial CBIR systems prefer to pose their queries in terms of key words. To help automate the indexing process, we represent images as sets of feature vectors of multiple types of abstract regions, which come from various segmentation processes. With this representation, we have developed two new algorithms to recognize classes of objects and concepts in outdoor photographic scenes. The semi-supervised EM-variant algorithm models each abstract region as a mixture of Gaussian distributions over its feature space. The more powerful generative/discriminative learning algorithm is a two-phase method. The generative phase normalizes the description length of images, which can have an arbitrary number of extracted features. In the discriminative phase, a classifier learns which images, as represented by this fixed-length description, contain the target object. We have tested our approaches by experimenting with several different data sets and combinations of features. Our results showed a significant improvement over the published results.

[1]  John R. Smith,et al.  Image Classification and Querying Using Composite Region Templates , 1999, Comput. Vis. Image Underst..

[2]  P. Hall,et al.  Data sharpening as a prelude to density estimation , 1999 .

[3]  Michael Brady,et al.  Saliency, Scale and Image Description , 2001, International Journal of Computer Vision.

[4]  Pietro Perona,et al.  Object class recognition by unsupervised scale-invariant learning , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[5]  Alex Pentland,et al.  Photobook: Content-based manipulation of image databases , 1996, International Journal of Computer Vision.

[6]  Gunther Wyszecki,et al.  Color Science: Concepts and Methods, Quantitative Data and Formulae, 2nd Edition , 2000 .

[7]  Shih-Fu Chang,et al.  Image and video search engine for the World Wide Web , 1997, Electronic Imaging.

[8]  Hiroshi Murase,et al.  Visual learning and recognition of 3-d objects from appearance , 2005, International Journal of Computer Vision.

[9]  Shih-Fu Chang,et al.  VisualSEEk: a fully automated content-based image query system , 1997, MULTIMEDIA '96.

[10]  Yi Li,et al.  Consistent line clusters for building recognition in CBIR , 2002, Object recognition supported by user interaction for service robots.

[11]  David A. Forsyth,et al.  Invariant Descriptors for 3D Object Recognition and Pose , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Amarnath Gupta,et al.  Virage image search engine: an open framework for image management , 1996, Electronic Imaging.

[13]  Yihong Gong.  Advancing content-based image retrieval by exploiting image color and region features , 1999, Multimedia Systems.

[14]  W. Seelen,et al.  Intensity and edge-based symmetry detection with an application to car-following , 1993 .

[15]  Jeff A. Bilmes,et al.  Object class recognition using images of abstract regions , 2004, ICPR 2004.

[16]  David A. Forsyth,et al.  Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary , 2002, ECCV.

[17]  Andreas Kuehnle,et al.  Symmetry-based recognition of vehicle rears , 1991, Pattern Recognit. Lett..

[18]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Jitendra Malik,et al.  Normalized Cuts and Image Segmentation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Pietro Perona,et al.  Unsupervised Learning of Models for Recognition , 2000, ECCV.

[21]  Linda G. Shapiro,et al.  A Flexible Image Database System for Content-Based Retrieval , 1999, Comput. Vis. Image Underst..

[22]  Linda G. Shapiro,et al.  3D Object Recognition and Pose with Relational Indexing , 2000, Comput. Vis. Image Underst..

[23]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Stan Sclaroff,et al.  Object recognition and categorization using modal matching , 1994, Proceedings of 1994 IEEE 2nd CAD-Based Vision Workshop.

[25]  A. H. Etemadi.  Robust segmentation of edge data , 1992 .

[26]  Jitendra Malik,et al.  Blobworld: A System for Region-Based Image Indexing and Retrieval , 1999, VISUAL.

[27]  Kyuseok Shim,et al.  WALRUS: a similarity retrieval algorithm for image databases , 1999, IEEE Transactions on Knowledge and Data Engineering.

[28]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[29]  James Ze Wang,et al.  Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Alberto Del Bimbo,et al.  Visual image retrieval by elastic deformation of object sketches , 1994, Proceedings of 1994 IEEE Symposium on Visual Languages.

[31]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  B. S. Manjunath,et al.  NeTra: A toolbox for navigating large image databases , 1997, Multimedia Systems.

[33]  R. Manmatha,et al.  Automatic image annotation and retrieval using cross-media relevance models , 2003, SIGIR.

[34]  David A. Forsyth,et al.  Body plans , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  David A. Forsyth,et al.  Finding Naked People , 1996, ECCV.

[36]  David A. Forsyth,et al.  Learning the semantics of words and pictures , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[37]  Thomas S. Huang,et al.  Supporting content-based queries over images in MARS , 1997, Proceedings of IEEE International Conference on Multimedia Computing and Systems.

[38]  Yizong Cheng,et al.  Mean Shift, Mode Seeking, and Clustering , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Thomas G. Dietterich,et al.  Solving the Multiple Instance Problem with Axis-Parallel Rectangles , 1997, Artif. Intell..

[40]  Shimon Ullman,et al.  Recognizing solid objects by alignment with an image , 1990, International Journal of Computer Vision.

[41]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  David A. Forsyth,et al.  Automatic Detection of Human Nudes , 1999, International Journal of Computer Vision.

[44]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Larry D. Hostetler,et al.  The estimation of the gradient of a density function, with applications in pattern recognition , 1975, IEEE Trans. Inf. Theory.