On 3D object retrieval benchmarking

AbstractThe continuous evolution of 3D computer graphics and the progress of 3D digitization systems resulted in a continuous increase in the available 3D content. The widespread use of 3D objects in diverse domains contributed on forming 3D object retrieval as an active research field. In order to objectively evaluate the performance of retrieval methodologies there is a need for objective benchmarking schemes. In this work, we provide a comprehensive overview of the state-of-the-art evaluation methodologies including not only the performance measures but also the corresponding benchmark datasets. Meaningful benchmark datasets are discussed while a detailed list of publicly available 3D model repositories is given organized in terms of application domains, content magnitude and data types.

[1]  Petros Daras,et al.  SHREC'09 Track: Structural Shape Retrieval on Watertight Models , 2009, 3DOR@Eurographics.

[2]  Paul Suetens,et al.  SHREC '11 Track: Shape Retrieval on Non-rigid 3D Watertight Meshes , 2011, 3DOR@Eurographics.

[3]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  George Pavlidis,et al.  3D Pottery content-based retrieval based on pose normalisation and segmentation , 2010 .

[5]  Frank B. ter Haar,et al.  SHREC 2009 - Shape Retrieval Contest , 2009, 3DOR@Eurographics.

[6]  Afzal Godil,et al.  A New Shape Benchmark for 3D Object Retrieval , 2008, ISVC.

[7]  Anestis Koutsoudis,et al.  3D Object Partial Matching Using Panoramic Views , 2013, ICIAP Workshops.

[8]  Pasquale Savino,et al.  Approximate similarity search in metric spaces using inverted files , 2008, Infoscale.

[9]  U. Hillenbrand,et al.  SHREC 2010 - Shape Retrieval Contest of Range Scans , 2010 .

[10]  George Pavlidis,et al.  Qp: A tool for generating 3D models of ancient Greek pottery , 2009 .

[11]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[12]  Daniela Giorgi,et al.  SHREC'12 Track: Stability on Abstract Shapes , 2012, 3DOR@Eurographics.

[13]  Philip Shilane Shape Distinction for 3D Object Retrieval , 2008 .

[14]  F. Precioso,et al.  3D Content-Based Retrieval in Artwork Databases , 2007, 2007 3DTV Conference.

[15]  Masaki Aono,et al.  A large-scale Shape Benchmark for 3D object retrieval: Toyohashi shape benchmark , 2012, Proceedings of The 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference.

[16]  Jovan Popović,et al.  Deformation transfer for triangle meshes , 2004, SIGGRAPH 2004.

[17]  Ellen M. Voorhees,et al.  The Philosophy of Information Retrieval Evaluation , 2001, CLEF.

[18]  Daniel A. Keim,et al.  Content-Based 3D Object Retrieval , 2007, IEEE Computer Graphics and Applications.

[19]  Thomas A. Funkhouser,et al.  The Princeton Shape Benchmark , 2004, Proceedings Shape Modeling Applications, 2004..

[20]  Karthik Ramani,et al.  Developing an engineering shape benchmark for CAD models , 2006, Comput. Aided Des..

[21]  Petros Daras,et al.  SHREC 2009 - Shape Retrieval Contest of Partial 3D Models | NIST , 2009 .

[22]  Remco C. Veltkamp,et al.  SHREC'10 Track: Large Scale Retrieval , 2010, 3DOR@Eurographics.

[23]  Ali Shokoufandeh,et al.  Retrieving articulated 3-D models using medial surfaces , 2008, Machine Vision and Applications.

[24]  Reinhard Klein,et al.  Shape retrieval using 3D Zernike descriptors , 2004, Comput. Aided Des..

[25]  Ioannis Pitas,et al.  The i3DPost Multi-View and 3D Human Action/Interaction Database , 2009, 2009 Conference for Visual Media Production.

[26]  Daniela Giorgi,et al.  SHape REtrieval Contest 2007: Watertight Models Track , 2007 .

[27]  Karthik Ramani,et al.  SHape REtrieval contest 2008: CAD models , 2008, 2008 IEEE International Conference on Shape Modeling and Applications.

[28]  Hao Wang,et al.  Perception-based shape retrieval for 3D building models , 2013 .

[29]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[30]  Bogdan J. Matuszewski,et al.  Hi4D-ADSIP 3-D dynamic facial articulation database , 2012, Image Vis. Comput..

[31]  Yiyu Yao,et al.  Evaluating information retrieval system performance based on user preference , 2010, Journal of Intelligent Information Systems.

[32]  Guillaume Lavoué,et al.  A comparative study of existing metrics for 3D-mesh segmentation evaluation , 2010, The Visual Computer.

[33]  Adrian Hilton,et al.  A FACS valid 3D dynamic action unit database with applications to 3D dynamic morphable facial modeling , 2011, 2011 International Conference on Computer Vision.

[34]  Lijun Yin,et al.  A high-resolution 3D dynamic facial expression database , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[35]  Ioannis Pratikakis,et al.  PANORAMA: A 3D Shape Descriptor Based on Panoramic Views for Unsupervised 3D Object Retrieval , 2010, International Journal of Computer Vision.

[36]  Hassen Drira,et al.  SHREC '11 Track: 3D Face Models Retrieval , 2011, 3DOR@Eurographics.

[37]  Afzal Godil,et al.  Benchmarks, performance evaluation and contests for 3D shape retrieval , 2010, PerMIS.

[38]  Ryutarou Ohbuchi,et al.  SHREC'10 Track: Generic 3D Warehouse , 2010, 3DOR@Eurographics.

[39]  Tsunenori Ishioka,et al.  Evaluation of criteria for information retrieval , 2003, Proceedings IEEE/WIC International Conference on Web Intelligence (WI 2003).

[40]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[41]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[42]  Tsunenori Ishioka,et al.  Improving heuristic function of cost-based abduction system using real-time heuristic search , 2004 .

[43]  Francoise J. Preteux,et al.  3D-shape-based retrieval within the MPEG-7 framework , 2001, IS&T/SPIE Electronic Imaging.

[44]  Reinhard Klein,et al.  A 3D Shape Benchmark for Retrieval and Automatic Classification of Architectural Data , 2009, 3DOR@Eurographics.

[45]  Ming Ouhyoung,et al.  On Visual Similarity Based 3D Model Retrieval , 2003, Comput. Graph. Forum.

[46]  Guillaume Lavoué,et al.  SHREC'12 Track: 3D Mesh Segmentation , 2012, 3DOR@Eurographics.

[47]  Ali Shokoufandeh,et al.  Retrieving Articulated 3-D Models Using Medial Surfaces and Their Graph Spectra , 2005, EMMCVPR.

[48]  J. G. Snodgrass,et al.  A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. , 1980, Journal of experimental psychology. Human learning and memory.

[49]  Fredric C. Gey,et al.  The relationship between recall and precision , 1994 .

[50]  Paul H. Lewis,et al.  SCULPTEUR: Multimedia Retrieval for Museums , 2004, CIVR.

[51]  Arjan Kuijper,et al.  Sketch-based 3D model retrieval using diffusion tensor fields of suggestive contours , 2010, ACM Multimedia.

[52]  Mark Goadrich,et al.  The relationship between Precision-Recall and ROC curves , 2006, ICML.

[53]  Ryutarou Ohbuchi,et al.  SHREC'10 Track: Range Scan Retrieval , 2010, 3DOR@Eurographics.

[54]  Antonios Danelakis,et al.  A survey on facial expression recognition in 3D video sequences , 2014, Multimedia Tools and Applications.