Simple object recognition based on spatial relations and visual features represented using irregular pyramids

Spatial relations among objects and object parts play a fundamental role in the human perception and understanding of images, thus becoming very relevant in the computational fields of object recognition, scene understanding and content-based image retrieval. In this work we propose a graph matching scheme that involves color, texture and shape features along with spatial descriptors to represent topological and orientation/directional relationships—which are obtained by means of combinatorial pyramids—in order to identify similar objects from a database. We also suggest a method for deciding which are the more useful levels in the hierarchy of segmentation for the recognition process. Our main objective is to prove that the combination of visual and spatial features is a promising road in order to improve the object recognition task. We performed experiments on two well known databases, COIL-100 and ETH-80 image sets, in order to evaluate the expressiveness of the proposed representation. These sets introduce challenges for simple object recognition in terms of view-point changes, and our results were comparable or superior than other state-of-the-art methods.

[1]  Yll Haxhimusa,et al.  Vision pyramids that do not grow too high , 2005, Pattern Recognit. Lett..

[2]  Thomas Martin Deserno,et al.  Content-based image retrieval by matching hierarchical attributed region adjacency graphs , 2004, SPIE Medical Imaging.

[3]  Kuniaki Uehara,et al.  Content-based Image Classification via Visual Learning , 2009 .

[4]  Joo-Hwee Lim,et al.  Visual graph modeling for scene recognition and mobile robot localization , 2010, Multimedia Tools and Applications.

[5]  Annette Morales-González,et al.  Assessing the Role of Spatial Relations for the Object Recognition Task , 2010, CIARP.

[6]  Yll Haxhimusa,et al.  Hierarchies Relating Topology and Geometry , 2006, Cognitive Vision Systems.

[7]  Shree K. Nayar,et al.  Spatial information in multiresolution histograms , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[8]  Bernt Schiele,et al.  Analyzing contour and appearance based methods for object categorization , 2003, CVPR 2003.

[9]  Ralf Hartmut Güting Dr.rer.nat.  An introduction to spatial database systems , 2005, The VLDB Journal.

[10]  Sargur N. Srihari,et al.  Binary Vector Dissimilarity Measures for Handwriting Identification , 2003, IS&T/SPIE Electronic Imaging.

[11]  W. Eric L. Grimson,et al.  Spatial template extraction for image retrieval by region matching , 2003, IEEE Trans. Image Process..

[12]  Aline Deruyver,et al.  Qualitative Spatial Relationships for Image Interpretation by Using Semantic Graph , 2007, GbRPR.

[13]  Luc Brun,et al.  Contraction kernels and combinatorial maps , 2003, Pattern Recognit. Lett..

[14]  Trevor Darrell,et al.  The Pyramid Match Kernel: Efficient Learning with Sets of Features , 2007, J. Mach. Learn. Res..

[15]  D. S. Guru,et al.  An Effective and Efficient Exact Match Retrieval Scheme for Symbolic Image Database Systems Based on Spatial Reasoning: A Logarithmic Search Time Approach , 2006, IEEE Transactions on Knowledge and Data Engineering.

[16]  Sergios Petridis,et al.  Classifying Images from Athletics Based on Spatial Relations , 2007 .

[17]  Yll Haxhimusa,et al.  Segmentation Graph Hierarchies , 2004, SSPR/SPR.

[18]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Raphaël Marée,et al.  Decision Trees and Random Subwindows for Object Recognition , 2005 .

[20]  Carlos Arturo Hernández-Gracidas,et al.  Markov Random Fields and Spatial Information to Improve Automatic Image Annotation , 2007, PSIVT.

[21]  Ching Y. Suen,et al.  Character Recognition Systems: A Guide for Students and Practitioners , 2007 .

[22]  Nobuyuki Morioka.  Learning Object Representations Using Sequential Patterns , 2008, Australasian Conference on Artificial Intelligence.

[23]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[24]  Yong Wang,et al.  Tensor Discriminant Analysis for View-based Object Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[25]  Jenny Benois-Pineau,et al.  Segmentation-based multi-class semantic object detection , 2012, Multimedia Tools and Applications.

[26]  Stepán Obdrzálek,et al.  Object Recognition using Local Affine Frames on Distinguished Regions , 2002, BMVC.

[27]  Adem Karahoca,et al.  Data Mining and Knowledge Discovery in Real Life Applications , 2009 .

[28]  Jayant Sharma,et al.  A Critical Comparison of the 4-Intersection and 9-Intersection Models for Spatial Relations: Formal Analysis* , 2003 .

[29]  Trevor Darrell,et al.  Pyramid Match Kernels: Discriminative Classification with Sets of Image Features (version 2) , 2006 .

[30]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[31]  Luc Brun,et al.  Introduction to Combinatorial Pyramids , 2000, Digital and Image Geometry.

[32]  Marko Heikkilä,et al.  Description of interest regions with local binary patterns , 2009, Pattern Recognit..

[33]  Edel B. García Reyes,et al.  A Multiple Substructure Matching Algorithm for Fingerprint Verification , 2007, CIARP.

[34]  Sandro Vega-Pons,et al.  Experimental Comparison of Orthogonal Moments as Feature Extraction Methods for Character Recognition , 2010, CIARP.

[35]  D. Gentner,et al.  Structure mapping in the comparison process. , 2000, The American journal of psychology.

[36]  Yann Gousseau,et al.  Adaptive image retrieval based on the spatial organization of colors , 2008, Comput. Vis. Image Underst..

[37]  Yll Haxhimusa,et al.  Effective Programming of Combinatorial Maps using COMA - A C++ Framework for Combinatorial Maps , 2006 .

[38]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[39]  B. Chandra Mohan,et al.  Content Based Image Retrieval Using Exact Legendre Moments and Support Vector Machine , 2010, ArXiv.

[40]  Hans-Hellmut Nagel,et al.  Cognitive Vision Systems, Sampling the Spectrum of Approaches [based on a Dagstuhl seminar] , 2006, Cognitive Vision Systems.

[41]  W. H. Tan,et al.  An efficient method for the retrieval of objects by topological relations in spatial database systems , 2003 .

[42]  Bernt Schiele,et al.  Analyzing appearance and contour based methods for object categorization , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[43]  Nicolas Tsapatsoulis,et al.  Classifying Images from Athletics Based on Spatial Relations , 2007, Second International Workshop on Semantic Media Adaptation and Personalization (SMAP 2007).

[44]  Cordelia Schmid,et al.  Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[45]  Trevor Darrell,et al.  The pyramid match kernel: discriminative classification with sets of image features , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[46]  Pablo Andrés Arbeláez,et al.  Finding Semantic Structures in Image Hierarchies Using Laplacian Graph Energy , 2010, ECCV.

[47]  Sami E. I. Baba,et al.  OBJECT CLASSIFICATION VIA GEOMETRICAL, ZERNIKE AND LEGENDRE MOMENTS , 2009 .

[48]  Spiros Skiadopoulos,et al.  Composing Cardinal Direction Relations , 2001, SSTD.

[49]  Luc Brun,et al.  Contains and inside relationships within combinatorial pyramids , 2006, Pattern Recognit..

[50]  Matti Pietikäinen,et al.  Block-Based Methods for Image Retrieval Using Local Binary Patterns , 2005, SCIA.

[51]  Thomas Martin Deserno,et al.  Hierarchical feature clustering for content-based retrieval in medical image databases , 2003, SPIE Medical Imaging.

[52]  Robert R. Sokal,et al.  A statistical method for evaluating systematic relationships , 1958 .