Object identification and retrieval from efficient image matching. Snap2Tell with the STOIC dataset

Traditional content based image retrieval attempts to retrieve images using syntactic features for a query image. Annotated image banks and Google allow the use of text to retrieve images. In this paper, we studied the task of using the content of an image to retrieve information in general. We describe the significance of object identification in an information retrieval paradigm that uses image set as intermediate means in indexing and matching. We also describe a unique Singapore Tourist Object Identification Collection with associated queries and relevance judgments for evaluating the new task and the need for efficient image matching using simple image features. We present comprehensive experimental evaluation on the effects of feature dimensions, context, spatial weightings, coverage of image indexes, and query devices on task performance. Lastly we describe the current system developed to support mobile image-based tourist information retrieval.

[1]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[2]  Georgios S. Paschos,et al.  Perceptually uniform color spaces for color texture analysis: an empirical evaluation , 2001, IEEE Trans. Image Process..

[3]  Gordon Dodds,et al.  A PDA-Based System for Recognizing Buildings from User-Supplied Images , 2003, Mobile HCI Workshop on Mobile and Ubiquitous Information Access.

[4]  Joo-Hwee Lim,et al.  Harnessing location-context for content-based services in vehicular systems , 2005, 2005 IEEE 61st Vehicular Technology Conference.

[5]  Jianping Fan,et al.  Automatic image annotation by using concept-sensitive salient objects for image content representation , 2004, SIGIR '04.

[6]  Ismail Haritaoglu,et al.  InfoScope : Link from Real World to Digital Information Space , 2001, UbiComp.

[7]  Alvy Ray Smith,et al.  Color gamut transform pairs , 1978, SIGGRAPH.

[8]  Gernot Kubin,et al.  On Interfaces for Mobile Information Retrieval , 2002, Mobile HCI.

[9]  Konrad Tollmar,et al.  IDeixis - Searching the Web with Mobile Images for Location-Based Information , 2004, Mobile HCI.

[10]  Moncef Gabbouj,et al.  Content-based image retrieval on mobile devices , 2005, IS&T/SPIE Electronic Imaging.

[11]  Jinxi Xu,et al.  Evaluation of an extraction-based approach to answering definitional questions , 2004, SIGIR '04.

[12]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[13]  Jonathon S. Hare,et al.  Content-based image retrieval using a mobile device as a novel interface , 2005, IS&T/SPIE Electronic Imaging.

[14]  Steven K. Feiner,et al.  A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment , 1997, Digest of Papers. First International Symposium on Wearable Computers.