Fast Image Classification for Monument Recognition

Content-based image classification is a wide research field that addresses the landmark recognition problem. Among the many classification techniques proposed, the k-nearest neighbor (kNN) is one of the most simple and widely used methods. In this article, we use kNN classification and landmark recognition techniques to address the problem of monument recognition in images. We propose two novel approaches that exploit kNN classification technique in conjunction with local visual descriptors. The first approach is based on a relaxed definition of the local feature based image to image similarity and allows standard kNN classification to be efficiently executed with the support of access methods for similarity search. The second approach uses kNN classification to classify local features rather than images. An image is classified evaluating the consensus among the classification of its local features. In this case, access methods for similarity search can be used to make the classification approach efficient. The proposed strategies were extensively tested and compared against other state-of-the-art alternatives in a monument and cultural heritage landmark recognition setting. The results proved the superiority of our approaches. An additional relevant contribution of this work is the exhaustive comparison of various types of local features and image matching solutions for recognition of monuments and cultural heritage related landmarks.

[1]  Felix Wu,et al.  Flickr-tag prediction using multi-modal fusion and meta information , 2013, MM '13.

[2]  V Korde,et al.  TEXT CLASSIFICATION AND CLASSIFIERS: A SURVEY , 2012 .

[3]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[4]  Richard I. Hartley,et al.  In Defense of the Eight-Point Algorithm , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[6]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[7]  Vincent Lepetit,et al.  Noname manuscript No. (will be inserted by the editor) Learning Real-Time Perspective Patch Rectification , 2022 .

[8]  Alexei A. Efros,et al.  IM2GPS: estimating geographic information from a single image , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Chokri Ben Amar,et al.  Classification improvement of local feature vectors over the KNN algorithm , 2011, Multimedia Tools and Applications.

[11]  Fabrizio Falchi,et al.  kNN based image classification relying on local feature similarity , 2010, SISAP.

[12]  Ze-Nian Li,et al.  Learning Sparse Features On-Line for Image Classification , 2011, ICIAR.

[13]  Adrian Popescu,et al.  MonuAnno: automatic annotation of georeferenced landmarks images , 2009, CIVR '09.

[14]  Panu Turcot,et al.  Better matching with fewer features: The selection of useful features in large database recognition problems , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[15]  Yihong Gong,et al.  Locality-constrained Linear Coding for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Bernt Schiele,et al.  Local features for object class recognition , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[17]  Fabrizio Falchi,et al.  Recognizing Landmarks Using Automated Classification Techniques: Evaluation of Various Visual Features , 2010, 2010 Second International Conferences on Advances in Multimedia.

[18]  Caetano Traina,et al.  Similarity Search and Applications , 2014, Lecture Notes in Computer Science.

[19]  Alexandros Nanopoulos,et al.  Nearest neighbors in high-dimensional data: the emergence and influence of hubs , 2009, ICML '09.

[20]  Guang-Zhong Yang,et al.  From images to scenes: Compressing an image cluster into a single scene model for place recognition , 2011, 2011 International Conference on Computer Vision.

[21]  Yang Song,et al.  Tour the world: Building a web-scale landmark recognition engine , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Trevor Darrell,et al.  The Pyramid Match Kernel: Efficient Learning with Sets of Features , 2007, J. Mach. Learn. Res..

[23]  Xiao Zhang,et al.  Efficient indexing for large scale visual search , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[24]  Eli Shechtman,et al.  In defense of Nearest-Neighbor based image classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[26]  Marc Sebban,et al.  A Survey on Metric Learning for Feature Vectors and Structured Data , 2013, ArXiv.

[27]  Fabrizio Falchi,et al.  Local Feature based Image Similarity Functions for kNN Classification , 2011, ICAART.

[28]  Tat-Seng Chua,et al.  Tour the world: Building a web-scale landmark recognition engine , 2009, CVPR.

[29]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Nicolás García-Pedrajas,et al.  Boosting k-nearest neighbor classifier by means of input space projection , 2009, Expert Syst. Appl..

[31]  Alexei A. Efros,et al.  Recognition by association via learning per-exemplar distances , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Simon J. D. Prince,et al.  Computer Vision: Models, Learning, and Inference , 2012 .

[33]  Hanan Samet,et al.  Foundations of multidimensional and metric data structures , 2006, Morgan Kaufmann series in data management systems.

[34]  Jan-Michael Frahm,et al.  Comparative Evaluation of Binary Features , 2012, ECCV.

[35]  Joo-Hwee Lim,et al.  Object Identification and Retrieval from Efficient Image Matching: Snap2Tell with the STOIC Dataset , 2005, AIRS.

[36]  Zhen Li,et al.  A Survey on Mobile Landmark Recognition for Information Retrieval , 2009, 2009 Tenth International Conference on Mobile Data Management: Systems, Services and Middleware.

[37]  Andrew Zisserman,et al.  Scene Classification Using a Hybrid Generative/Discriminative Approach , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Florent Perronnin,et al.  Fisher Kernels on Visual Vocabularies for Image Categorization , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Cordelia Schmid,et al.  Aggregating local descriptors into a compact image representation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[40]  Luc Van Gool,et al.  Naive Bayes Image Classification: Beyond Nearest Neighbors , 2012, ACCV.

[41]  Claudio Gennaro,et al.  Large Scale Image Retrieval Using Vectors of Locally Aggregated Descriptors , 2013, ERCIM News.

[42]  Martial Hebert,et al.  The optimal distance measure for object detection , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[43]  AmatoGiuseppe,et al.  Fast Image Classification for Monument Recognition , 2015 .

[44]  Sahibsingh A. Dudani The Distance-Weighted k-Nearest-Neighbor Rule , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[45]  Pavel Zezula,et al.  Similarity Search: The Metric Space Approach (Advances in Database Systems) , 2005 .

[46]  Alexandros Nanopoulos,et al.  Hubs in Space: Popular Nearest Neighbors in High-Dimensional Data , 2010, J. Mach. Learn. Res..

[47]  Jitendra Malik,et al.  SVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[48]  Claudio Gennaro,et al.  Geometric consistency checks for kNN based image classification relying on local features , 2011, SISAP.

[49]  Cordelia Schmid,et al.  A performance evaluation of local descriptors , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Pierre Tirilly,et al.  Distances and weighting schemes for bag of visual words image retrieval , 2010, MIR '10.

[51]  Joo-Hwee Lim,et al.  Object identification and retrieval from efficient image matching. Snap2Tell with the STOIC dataset , 2007, Inf. Process. Manag..

[52]  Richard Nock,et al.  Boosting k-Nearest Neighbors Classification , 2013, Advanced Topics in Computer Vision.

[53]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[54]  Michael Isard,et al.  Object retrieval with large vocabularies and fast spatial matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Richard I. Hartley,et al.  In defence of the 8-point algorithm , 1995, Proceedings of IEEE International Conference on Computer Vision.

[56]  Dunja Mladenic,et al.  Hubness-based fuzzy measures for high-dimensional k-nearest neighbor classification , 2011, International Journal of Machine Learning and Cybernetics.

[57]  Jitendra Malik,et al.  Image Retrieval and Classification Using Local Distance Functions , 2006, NIPS.

[58]  F. Falchi,et al.  Image classification via adaptive ensembles of descriptor-specific classifiers , 2010, Pattern Recognition and Image Analysis.

[59]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[60]  Pavel Zezula,et al.  Similarity Search - The Metric Space Approach , 2005, Advances in Database Systems.

[61]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[62]  David Zhang,et al.  On kernel difference-weighted k-nearest neighbor classification , 2008, Pattern Analysis and Applications.

[63]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .