Mobile Museum Guidance Using Relational Multi-Image Classification

In this paper we present a multi-image classification technique for mobile phones that is supported by relational reasoning. Users capture a sequence of images employing a simple near-far camera movement. After classifying distinct keyframes using a nearest-neighbor approach the corresponding database images are only considered for a majority voting if they exhibit similar near-far inter-image relations to the captured keyframes. In the context of PhoneGuide, our adaptive mobile museum guidance system, a user study revealed that our multi-image classification technique leads to significantly higher classification rates than single image classification. Furthermore, when using near-far image relations, less keyframes are sufficient for classification. This increases the overall classification speed of our approach by up to 35%.

[1]  Luc Van Gool,et al.  Fast indexing for image retrieval based on local appearance with re-ranking , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[2]  Mohamed S. Kamel,et al.  Decision fusion in neural network ensembles , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[3]  Oliver Bimber,et al.  Adaptive training of video sets for image recognition on mobile phones , 2009, Personal and Ubiquitous Computing.

[4]  Hugh E. Williams,et al.  Are two pictures better than one? , 2001, Proceedings 12th Australasian Database Conference. ADC 2001.

[5]  Ada Wai-Chee Fu,et al.  Dynamic vp-tree indexing for n-nearest neighbor search given pair-wise distances , 2000, The VLDB Journal.

[6]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[7]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[8]  Oliver Bimber,et al.  Localization and Classification through Adaptive Pathway Analysis , 2012, IEEE Pervasive Computing.

[9]  Hermann Ney,et al.  Features for image retrieval: an experimental comparison , 2008, Information Retrieval.

[10]  Bryan Pardo,et al.  Speeding Melody Search With Vantage Point Trees , 2008, ISMIR.

[11]  James C. French,et al.  Improving Image Retrieval Effectiveness via Multiple Queries , 2003, MMDB '03.

[12]  Santanu Chaudhury,et al.  Multiple Exemplar-Based Facial Image Retrieval Using Independent Component Analysis , 2006, IEEE Transactions on Image Processing.

[13]  Peter N. Yianilos,et al.  Data structures and algorithms for nearest neighbor search in general metric spaces , 1993, SODA '93.

[14]  Lucas Paletta,et al.  Visual object detection from mobile phone imagery for context awareness , 2005, Mobile HCI.

[15]  Thomas S. Huang,et al.  Relevance feedback in image retrieval: A comprehensive review , 2003, Multimedia Systems.

[16]  Andrea Salgian,et al.  Appearance-based object recognition using multiple views , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[17]  Oliver Bimber,et al.  Mobile Phone-Enabled Museum Guidance with Adaptive Classification , 2008, IEEE Computer Graphics and Applications.

[18]  Oliver Bimber,et al.  Enabling Mobile Phones To Support Large-Scale Museum Guidance , 2007, IEEE MultiMedia.

[19]  Lucas Paletta,et al.  A Mobile Vision System for Urban Detection with Informative Local Descriptors , 2006, Fourth IEEE International Conference on Computer Vision Systems (ICVS'06).

[20]  D. N. F. Awang Iskandar,et al.  Content-based Image Retrieval Using Image Regions as Query Examples , 2008, ADC.

[21]  Joo-Hwee Lim,et al.  Scene Recognition with Camera Phones for Tourist Information Access , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[22]  Bernd Girod,et al.  Outdoors augmented reality on mobile phone using loxel-based visual feature organization , 2008, MIR '08.

[23]  Oliver Bimber,et al.  Phone-to-phone communication for adaptive image classification , 2008, MoMM.

[24]  Dieter Schmalstieg,et al.  Pose tracking from natural features on mobile phones , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[25]  Niels Henze,et al.  What is That? Object Recognition from Natural Features on a Mobile Phone , 2009 .

[26]  Natasha Gelfand,et al.  SURFTrac: Efficient tracking and continuous object recognition using local feature descriptors , 2009, CVPR.

[27]  Jonathon S. Hare,et al.  Content-based image retrieval using a mobile device as a novel interface , 2005, IS&T/SPIE Electronic Imaging.

[28]  Hugh E. Williams,et al.  Are two pictures better than one? , 2001, Proceedings 12th Australasian Database Conference. ADC 2001.

[29]  Luc Van Gool,et al.  Interactive museum guide , 2005 .