Scalable landmark recognition using EXTENT

We have proposed the EXTENT system for automated photograph annotation using image content and context analysis. A key component of EXTENT is a Landmark recognition system called LandMarker. In this paper, we present the architecture of LandMarker. The content of a query photograph is analyzed and compared against a database of sample landmark images, to recognize any landmarks it contains. An algorithm is presented for comparing a query image with a sample image. Context information may be used to assist landmark recognition. Also, we show how LandMarker deals with scalability to allow recognition of a large number of landmarks. We have implemented a prototype of the system, and present empirical results on a large dataset.

[1]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[2]  Edward Y. Chang,et al.  EXTENT: fusing context, content, and semantic ontology for photo annotation , 2005, CVDB '05.

[3]  Mor Naaman,et al.  Context data in geo-referenced digital photo collections , 2004, MULTIMEDIA '04.

[4]  Mor Naaman,et al.  From Where to What: Metadata Sharing for Digital Photographs with Geographic Coordinates , 2003, OTM.

[5]  Piotr Indyk,et al.  Similarity Search in High Dimensions via Hashing , 1999, VLDB.

[6]  Yan Ke,et al.  An efficient parts-based near-duplicate and sub-image retrieval system , 2004, MULTIMEDIA '04.

[7]  Nicu Sebe,et al.  Fast spatial pattern discovery integrating boosting with constellations of contextual descriptors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[8]  Cordelia Schmid,et al.  Pattern recognition with local invariant features , 2005 .

[9]  Kai Li,et al.  Image similarity search with compact data structures , 2004, CIKM '04.

[10]  Trevor Darrell,et al.  Efficient image matching with distributions of local invariant features , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Nir Friedman,et al.  Learning Bayesian Networks from Data , 2001 .

[12]  Roger Weber,et al.  Efficient region-based image retrieval , 2003, CIKM '03.

[13]  Yan Ke,et al.  Efficient Near-duplicate Detection and Sub-image Retrieval , 2004 .

[14]  Jitendra Malik,et al.  Blobworld: A System for Region-Based Image Indexing and Retrieval , 1999, VISUAL.

[15]  Ross D. Shachter,et al.  Decision-Theoretic Foundations for Causal Reasoning , 1995, J. Artif. Intell. Res..

[16]  P. Cheng,et al.  Assessing interactive causal influence. , 2004, Psychological review.

[17]  Diomidis Spinellis.  Position-Annotated Photographs: A Geotemporal Web , 2003, IEEE Pervasive Comput..

[18]  Edward Y. Chang,et al.  Support vector machine active learning for image retrieval , 2001, MULTIMEDIA '01.

[19]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[20]  Nicole Immorlica,et al.  Locality-sensitive hashing scheme based on p-stable distributions , 2004, SCG '04.

[21]  Simon King,et al.  From context to content: leveraging context to infer media metadata , 2004, MULTIMEDIA '04.

[22]  Edward Y. Chang,et al.  SVM binary classifier ensembles for image classification , 2001, CIKM '01.

[23]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[24]  Anind K. Dey,et al.  Understanding and Using Context , 2001, Personal and Ubiquitous Computing.

[25]  Ilaria Bartolini,et al.  A sound algorithm for region-based image retrieval using an index , 2000, Proceedings 11th International Workshop on Database and Expert Systems Applications.

[26]  David A. Forsyth,et al.  Learning the semantics of words and pictures , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.