A search engine for 3D models

As the number of 3D models available on the Web grows, there is an increasing need for a search engine to help people find them. Unfortunately, traditional text-based search techniques are not always effective for 3D data. In this article, we investigate new shape-based search methods. The key challenges are to develop query methods simple enough for novice users and matching algorithms robust enough to work for arbitrary polygonal models. We present a Web-based search engine system that supports queries based on 3D sketches, 2D sketches, 3D models, and/or text keywords. For the shape-based queries, we have developed a new matching algorithm that uses spherical harmonics to compute discriminating similarity measures without requiring repair of model degeneracies or alignment of orientations. It provides 46 to 245% better performance than related shape-matching methods during precision--recall experiments, and it is fast enough to return query results from a repository of 20,000 models in under a second. The net result is a growing interactive index of 3D models available on the Web (i.e., a Google for 3D models).

[1]  Gerard Salton,et al.  The SMART Retrieval System—Experiments in Automatic Document Processing , 1971 .

[2]  Gerard Salton,et al.  The SMART Retrieval System , 1971 .

[3]  Rudy de Reyna How to Draw What You See , 1972 .

[4]  Ralph Roskies,et al.  Fourier Descriptors for Plane Closed Curves , 1972, IEEE Transactions on Computers.

[5]  Matthew Goldstein,et al.  Kn -nearest Neighbor Classification , 1972, IEEE Trans. Inf. Theory.

[6]  David G. Stork,et al.  Pattern Classification , 1973 .

[7]  Ian T. Young,et al.  An Analysis Technique for Biological Shape. I , 1974, Inf. Control..

[8]  Robert C. Bolles,et al.  Parametric Correspondence and Chamfer Matching: Two New Techniques for Image Matching , 1977, IJCAI.

[9]  Martin F. Porter,et al.  An algorithm for suffix stripping , 1997, Program.

[10]  Rama Chellappa,et al.  Stochastic models for closed boundary analysis: Representation and reconstruction , 1981, IEEE Trans. Inf. Theory.

[11]  Charles C. Tappert,et al.  Cursive Script Recognition by Elastic Matching , 1982, IBM J. Res. Dev..

[12]  Berthold K. P. Horn Extended Gaussian images , 1984, Proceedings of the IEEE.

[13]  Wen-Hsiang Tsai,et al.  Attributed String Matching with Merging for Shape Recognition , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Ramesh C. Jain,et al.  Three-dimensional object recognition , 1985, CSUR.

[15]  Bir Bhanu,et al.  CAD-based robot vision , 1987 .

[16]  Andrew P. Witkin,et al.  Energy constraints on parameterized models , 1987, SIGGRAPH.

[17]  Yehezkel Lamdan,et al.  Geometric Hashing: A General And Efficient Model-based Recognition Scheme , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[18]  Yehezkel Lamdan,et al.  Affine invariant model-based object recognition , 1990, IEEE Trans. Robotics Autom..

[19]  William Grimson,et al.  Object recognition by computer - the role of geometric constraints , 1991 .

[20]  Wesley E. Snyder,et al.  Application of Affine-Invariant Fourier Descriptors to Recognition of 3-D Objects , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Ruzena Bajcsy,et al.  Recovery of Parametric Models from Range Images: The Case for Superquadrics with Global Deformations , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Dimitris N. Metaxas,et al.  Dynamic 3D models with local and global deformations: deformable superquadrics , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[23]  Esther M. Arkin,et al.  An efficiently computable metric for comparing polygonal shapes , 1991, SODA '90.

[24]  Alex Pentland,et al.  Closed-Form Solutions for Physically Based Shape Modeling and Recognition , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Alex Pentland,et al.  Closed-form solutions for physically-based shape modeling and recognition , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Gérard G. Medioni,et al.  Object modeling by registration of multiple range images , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[27]  U. Grenander,et al.  Structural Image Restoration through Deformable Templates , 1991 .

[28]  G. Miller WordNet: A Lexical Database for English , 1992, HLT.

[29]  Gérard G. Medioni,et al.  Object modelling by registration of multiple range images , 1992, Image Vis. Comput..

[30]  Andrew Zisserman,et al.  Geometric invariance in computer vision , 1992 .

[31]  Hongmei Wang,et al.  Contour shape description based on an arch height function , 1992, Pattern Recognit..

[32]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[33]  Katsushi Ikeuchi,et al.  Shape representation and image segmentation using deformable surfaces , 1992, Image Vis. Comput..

[34]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Neil A. Thacker,et al.  Pairwise representations of shape , 1992, [1992] Proceedings. 11th IAPR International Conference on Pattern Recognition.

[36]  Richard J. Prokop,et al.  A survey of moment-based techniques for unoccluded object representation and recognition , 1992, CVGIP Graph. Model. Image Process..

[37]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Katsushi Ikeuchi,et al.  A spherical representation for the recognition of curved objects , 1993, 1993 (4th) International Conference on Computer Vision.

[39]  Hiroshi Murase,et al.  Learning and recognition of 3D objects from appearance , 1993, [1993] Proceedings IEEE Workshop on Qualitative Vision.

[40]  Jake K. Aggarwal,et al.  Model-based object recognition in dense-range images—a review , 1993, CSUR.

[41]  Martin D. Levine,et al.  Recovering parametric geons from multiview range data , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Alessandro Verri,et al.  On the recognition of the alphabet of the sign language through size functions , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[43]  Arthur R. Pope Model-Based Object Recognition - A Survey of Recent Research , 1994 .

[44]  Marc Levoy,et al.  Zippered polygon meshes from range images , 1994, SIGGRAPH.

[45]  Paul J. Besl Triangles as a Primary Representation , 1994, Object Representation in Computer Vision.

[46]  Emanuele Trucco,et al.  Geometric Invariance in Computer Vision , 1995 .

[47]  Neil A. Thacker,et al.  Robust Recognition of Scaled Shapes using Pairwise Geometric Histograms , 1995, BMVC.

[48]  Alex Pentland,et al.  Modal Matching for Correspondence and Recognition , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  David Salesin,et al.  Fast multiresolution image querying , 1995, SIGGRAPH.

[50]  Katsushi Ikeuchi,et al.  Recent Progress in CAD-Based Vision , 1995, Comput. Vis. Image Underst..

[51]  Michael Stonebraker,et al.  Chabot: Retrieval from a Relational Database of Images , 1995, Computer.

[52]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[53]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[54]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[55]  Anil K. Jain,et al.  Object Matching Using Deformable Templates , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[56]  Subodh Kumar,et al.  Repairing CAD models , 1997, Proceedings. Visualization '97 (Cat. No. 97CB36155).

[57]  Mark A. Ganter,et al.  Skeleton-based modeling operations on solids , 1997, SMA '97.

[58]  Neil A. Thacker,et al.  Optimal Pairwise Geometric Histograms , 1997, BMVC.

[59]  Hans-Peter Kriegel,et al.  S3: similarity search in CAD database systems , 1997, SIGMOD '97.

[60]  T. M. Murali,et al.  Consistent solid and boundary representations from arbitrary polygonal data , 1997, SI3D.

[61]  Michael E. Lesk,et al.  Practical Digital Libraries: Books, Bytes, and Bucks , 1997 .

[62]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[63]  Shock graphs and shape matching , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[64]  Piotr Indyk,et al.  Approximate nearest neighbors: towards removing the curse of dimensionality , 1998, STOC '98.

[65]  Hans-Peter Kriegel,et al.  Similarity Search in 3D Protein Databases , 1998, German Conference on Bioinformatics.

[66]  Martial Hebert,et al.  Efficient multiple model recognition in cluttered 3-D scenes , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[67]  Leonidas J. Guibas,et al.  A metric for distributions with applications to image databases , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[68]  Gabriel Taubin,et al.  Converting sets of polygons to manifold surfaces by cutting and stitching , 1998, Proceedings Visualization '98 (Cat. No.98CB36276).

[69]  Sven Loncaric,et al.  A survey of shape analysis techniques , 1998, Pattern Recognit..

[70]  Hans-Peter Kriegel,et al.  3D Shape Histograms for Similarity Search and Classification in Spatial Databases , 1999, SSD.

[71]  Satoshi Matsuoka,et al.  Teddy: A Sketching Interface for 3D Freeform Design , 1999, SIGGRAPH Courses.

[72]  Dongmei Zhang,et al.  Harmonic maps and their applications in surface matching , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[73]  Ali Shokoufandeh,et al.  Indexing using a spectral encoding of topological structure , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[74]  James Scott,et al.  Fast polygon mesh querying by example , 1999, SIGGRAPH '99.

[75]  Jules Bloomenthal,et al.  Skeletal methods of shape manipulation , 1999, Proceedings Shape Modeling International '99. International Conference on Shape Modeling and Applications.

[76]  Jonathan Foote,et al.  An overview of audio information retrieval , 1999, Multimedia Systems.

[77]  Hans-Peter Kriegel,et al.  Nearest Neighbor Classification in 3D Protein Databases , 1999, ISMB.

[78]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[79]  Marc Rioux,et al.  Nefertiti: A tool for 3-D shape databases management , 1999 .

[80]  Grégoire Malandain,et al.  Structural Object Matching , 2000 .

[81]  Jitendra Malik,et al.  Shape contexts enable efficient retrieval of similar shapes , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[82]  Jitendra Malik,et al.  Matching Shapes , 2001, ICCV.

[83]  Taku Komura,et al.  Topology matching for fully automatic similarity estimation of 3D shapes , 2001, SIGGRAPH.

[84]  Dietmar Saupe,et al.  3D Model Retrieval with Spherical Harmonics and Moments , 2001, DAGM-Symposium.

[85]  Anshuman Razdan,et al.  Advances in geometric modeling and feature extraction on pots , 2001 .

[86]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[87]  M. T. Suzuki,et al.  A Web-based retrieval system for 3D polygonal models , 2001, Proceedings Joint 9th IFSA World Congress and 20th NAFIPS International Conference (Cat. No. 01TH8569).

[88]  Dietmar Saupe,et al.  Tools for 3D-object retrieval: Karhunen-Loeve transform and spherical harmonics , 2001, 2001 IEEE Fourth Workshop on Multimedia Signal Processing (Cat. No.01TH8564).

[89]  Remco C. Veltkamp,et al.  Shape matching: similarity measures and algorithms , 2001, Proceedings International Conference on Shape Modeling and Applications.

[90]  Hans-Peter Kriegel,et al.  State-of-the-Art in Content-Based Image and Video Retrieval , 2001, Computational Imaging and Vision.

[91]  Bernard Chazelle,et al.  Matching 3D models with shape distributions , 2001, Proceedings International Conference on Shape Modeling and Applications.

[92]  Sethuraman Panchanathan,et al.  A 3D Digital Library System: Capture, Analysis, Query, and Display , 2002 .

[93]  Michael Elad,et al.  Content based retrieval of VRML objects: an iterative and interactive approach , 2002 .

[94]  Vittorio Castelli,et al.  Image Databases: Search and Retrieval of Digital Imagery , 2002 .

[95]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[96]  Hiroshi Murase,et al.  Visual learning and recognition of 3-d objects from appearance , 2005, International Journal of Computer Vision.