Automatic Shape Expansion with Verification to Improve 3D Retrieval, Classification and Matching

The goal of this paper is to retrieve 3D object models from a database, that are similar to a single 3D object model, given as a query. The system has no prior models of any object class and is class-generic. The approach is fully automated and unsupervised. The main contribution of the paper is to improve the quality of such 3D shape retrieval, through query verification and query expansion. These are part of a cascaded, two-stage system: (i) Verification: after a first inexpensive and coarse retrieval step that uses a standard Bag-of-Words (BoW) over quantized local features, a fast but effective spatial layout verification of those words is used to prune the initial search results. (ii) Expansion: a new BoW query is issued on the basis of an expanded set of query shapes that, next to the original query, also includes the positively verified results of (i). We perform comprehensive evaluation and show improved performance. As an additional novelty, we show the usefulness of the query expansion on shape classification with limited training data and shape matching, domains in which it has not been used before. The experiments were performed on a variety of state-of-the-art datasets.

[1]  N. Mitra,et al.  Exploration of continuous variability in collections of 3D shapes , 2011, SIGGRAPH 2011.

[2]  Luc Van Gool,et al.  Class-specific 3D localization using constellations of object parts , 2011, BMVC.

[3]  Petros Daras,et al.  SHREC 2009 - Shape Retrieval Contest of Partial 3D Models | NIST , 2009 .

[4]  Tsuhan Chen,et al.  Feature space warping: an approach to relevance feedback , 2002, Proceedings. International Conference on Image Processing.

[5]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[6]  Andrew Zisserman,et al.  Efficient image retrieval for 3D structures , 2010, BMVC.

[7]  Michael Isard,et al.  Object retrieval with large vocabularies and fast spatial matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Ko Nishino,et al.  Scale-hierarchical 3D object recognition in cluttered scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Bernt Schiele,et al.  Robust Object Detection with Interleaved Categorization and Segmentation , 2008, International Journal of Computer Vision.

[10]  Anil K. Jain,et al.  A modified Hausdorff distance for object matching , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[11]  Leonidas J. Guibas,et al.  One Point Isometric Matching with the Heat Kernel , 2010, Comput. Graph. Forum.

[12]  Rong Jin,et al.  Active multiple kernel learning for interactive 3D object retrieval systems , 2011, ACM Trans. Interact. Intell. Syst..

[13]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Luc Van Gool,et al.  Hough Transform and 3D SURF for Robust Three Dimensional Classification , 2010, ECCV.

[15]  Jiri Matas,et al.  Total recall II: Query expansion revisited , 2011, CVPR 2011.

[16]  Andrew Zisserman,et al.  Geometric LDA: A Generative Model for Particular Object Discovery , 2008, BMVC.

[17]  Sunil Prabhakar,et al.  A Reconfigurable 3D Engineering Shape Search System: Part II — Database Indexing, Retrieval, and Clustering , 2003 .

[18]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[19]  Ioannis Pratikakis,et al.  Relevance Feedback in Content-based 3D Object Retrieval A Comparative Study , 2008 .

[20]  Bastian Leibe,et al.  Interleaved Object Categorization and Segmentation , 2003, BMVC.

[21]  Roland Wagner,et al.  Multi-Feature Integration with Relevance Feedback on 3D Model Similarity Retrieval , 2006, J. Mobile Multimedia.

[22]  Cordelia Schmid,et al.  Improving Bag-of-Features for Large Scale Image Search , 2010, International Journal of Computer Vision.

[23]  Thomas A. Funkhouser,et al.  The Princeton Shape Benchmark , 2004, Proceedings Shape Modeling Applications, 2004..

[24]  Michael Isard,et al.  Descriptor Learning for Efficient Retrieval , 2010, ECCV.

[25]  Leonidas J. Guibas,et al.  A Condition Number for Non‐Rigid Shape Matching , 2011, Computer Graphics Forum.

[26]  Vladimir G. Kim,et al.  Blended intrinsic maps , 2011, SIGGRAPH 2011.

[27]  Stephen DiVerdi,et al.  Exploring collections of 3D models using fuzzy correspondences , 2012, ACM Trans. Graph..

[28]  Michael Elad,et al.  Content based retrieval of VRML objects: an iterative and interactive approach , 2002 .

[29]  Leonidas J. Guibas,et al.  A concise and provably informative multi-scale signature based on heat diffusion , 2009 .

[30]  David G. Lowe,et al.  Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration , 2009, VISAPP.

[31]  Tom Minka,et al.  Interactive learning with a "Society of Models" , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[32]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Alexander M. Bronstein,et al.  Numerical Geometry of Non-Rigid Shapes , 2009, Monographs in Computer Science.

[34]  Cordelia Schmid,et al.  A contextual dissimilarity measure for accurate and efficient image search , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Ramakant Nevatia,et al.  Detection of multiple, partially occluded humans in a single image by Bayesian combination of edgelet part detectors , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[36]  Luc Van Gool,et al.  Hello neighbor: Accurate object retrieval with k-reciprocal nearest neighbors , 2011, CVPR 2011.

[37]  Maarten Vergauwen,et al.  Web-based 3D Reconstruction Service , 2006, Machine Vision and Applications.

[38]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[39]  Jiri Matas,et al.  Learning a Fine Vocabulary , 2010, ECCV.

[40]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[41]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Daniela Giorgi,et al.  3D relevance feedback via multilevel relevance judgements , 2010, The Visual Computer.

[43]  Szymon Rusinkiewicz,et al.  Modeling by example , 2004, SIGGRAPH 2004.

[44]  Dietmar Saupe,et al.  3D Shape Descriptor Based on 3D Fourier Transform , 2001 .

[45]  Andrea Fusiello,et al.  A Bag of Words Approach for 3D Object Categorization , 2009, MIRAGE.

[46]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[47]  Miles Efron,et al.  Query expansion and dimensionality reduction: Notions of optimality in Rocchio relevance feedback and latent semantic indexing , 2008, Inf. Process. Manag..

[48]  Leonidas J. Guibas,et al.  Shape google: Geometric words and expressions for invariant shape retrieval , 2011, TOGS.