3D model retrieval via single image based on feature mapping

With the development of manufacture, more and more 3D models are generated by users and many differnet factories. 3D model retrieval has been receiving more and more attention in computer vision and the field of data analysis. In this paper, we propose a novel 3D model retrieval algorithm by cross-modal feature mapping (CMFM), which utilize one single image as query information to address 3D model retrieval problem. Specifically, in this paper, we first proposed to leverage 2D image to handle 3d model retrieval problem, which is one new problem in this field. The proposed feature learning method can benefit: 1) avoiding the interference of query image recorded by different visual sensor; 2) handling cross-modal data retrieval by simple computer vision technologies, which can guarantee the performance of retrieval and also control that the retrieval time hold a low level; 3) the low complexity of this method can guarantee that this method can be applied in many fields. Finally, we validate the retrieval method on three popular datasets. Extensive comparison experiments show the superiority of the proposed mehtod. To the best of our knowledge, it is the first method to handle 3D model retreival based on one single 2D image.

[1]  Ryutarou Ohbuchi,et al.  Scale-weighted dense bag of visual features for 3D model retrieval from a partial view 3D model , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[2]  Weizhi Nie,et al.  Clique-graph matching by preserving global & local structure , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Ryutarou Ohbuchi,et al.  Salient local visual features for shape-based 3D model retrieval , 2008, 2008 IEEE International Conference on Shape Modeling and Applications.

[4]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[5]  A. Ben Hamza,et al.  Reeb graph path dissimilarity for 3D object matching and retrieval , 2011, The Visual Computer.

[6]  Zhiwen Liu,et al.  Boosting 3D model retrieval with class vocabularies and distance vector revision , 2015, TENCON 2015 - 2015 IEEE Region 10 Conference.

[7]  Bruce G. Baumgart,et al.  Geometric modeling for computer vision. , 1974 .

[8]  Weizhi Nie,et al.  Convolutional deep learning for 3D object retrieval , 2017, Multimedia Systems.

[9]  Yue Gao,et al.  3D model retrieval using weighted bipartite graph matching , 2011, Signal Process. Image Commun..

[10]  Inderjit S. Dhillon,et al.  Information-theoretic metric learning , 2006, ICML '07.

[11]  Ming Ouhyoung,et al.  On Visual Similarity Based 3D Model Retrieval , 2003, Comput. Graph. Forum.

[12]  Hans-Peter Kriegel,et al.  3D Shape Histograms for Similarity Search and Classification in Spatial Databases , 1999, SSD.

[13]  Tobias Isenberg,et al.  3D Shape Matching Using Skeleton Graphs , 2004, SimVis.

[14]  BENJAMIN BUSTOS,et al.  Feature-based similarity search in 3D object databases , 2005, CSUR.

[15]  King-Sun Fu,et al.  Shape Discrimination Using Fourier Descriptors , 1977, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Dong Du,et al.  A real-time object tracking and image stabilization system for photographing in vibration environment using OpenTLD algorithm , 2016, 2016 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO).

[17]  Yoshihisa Shinagawa,et al.  Automatic 3-D grayscale volume matching and shape analysis , 2006, IEEE Transactions on Information Technology in Biomedicine.

[18]  Dmitry Vavilov,et al.  Perspectives of stereo 3D TV applications development , 2010, 2010 6th Central and Eastern European Software Engineering Conference (CEE-SECR).

[19]  Hongxun Yao,et al.  View-based 3D object retrieval via multi-modal graph learning , 2015, Signal Process..

[20]  Anni Cai,et al.  Enhanced and hierarchical structure algorithm for data imbalance problem in semantic extraction under massive video dataset , 2012, Multimedia Tools and Applications.

[21]  D. Jacobs,et al.  Bypassing synthesis: PLS for face recognition with pose, low-resolution and sketch , 2011, CVPR 2011.

[22]  Petros Daras,et al.  A 3D Shape Retrieval Framework Supporting Multimodal Queries , 2010, International Journal of Computer Vision.

[23]  Yue Gao,et al.  3-D Object Retrieval and Recognition With Hypergraph Analysis , 2012, IEEE Transactions on Image Processing.

[24]  Tosiyasu L. Kunii,et al.  Constructing a Reeb graph automatically from cross sections , 1991, IEEE Computer Graphics and Applications.

[25]  Chen-Ta Hsieh,et al.  3D model retrieval using multiple features and manifold ranking , 2015, 2015 8th International Conference on Ubi-Media Computing (UMEDIA).

[26]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[27]  Whoi-Yul Kim,et al.  A region-based shape descriptor using Zernike moments , 2000, Signal Process. Image Commun..

[28]  William C. Regli,et al.  Managing digital libraries for computer-aided design , 2000, Comput. Aided Des..

[29]  Yue Gao,et al.  Camera Constraint-Free View-Based 3-D Object Retrieval , 2012, IEEE Transactions on Image Processing.

[30]  Bernard Chazelle,et al.  Shape distributions , 2002, TOGS.

[31]  Bernt Schiele,et al.  Analyzing appearance and contour based methods for object categorization , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[32]  William C. Regli,et al.  Using shape distributions to compare solid models , 2002, SMA '02.

[33]  Weizhi Nie,et al.  3D object retrieval based on Spatial+LDA model , 2015, Multimedia Tools and Applications.

[34]  Jean Ponce,et al.  A tensor-based algorithm for high-order graph matching , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Daniel Cremers,et al.  The wave kernel signature: A quantum mechanical approach to shape analysis , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[36]  H. Zhang,et al.  Multi-perspective and multi-modality joint representation and recognition model for 3D action recognition , 2015, Neurocomputing.

[37]  Ko Nishino,et al.  Scale-Dependent 3D Geometric Features , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[38]  Bernard Chazelle,et al.  Matching 3D models with shape distributions , 2001, Proceedings International Conference on Shape Modeling and Applications.

[39]  Uwe Stilla,et al.  Active learning approach to detecting standing dead trees from ALS point clouds combined with aerial infrared imagery , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[40]  Mohamed Daoudi,et al.  A Bayesian 3-D Search Engine Using Adaptive Views Clustering , 2007, IEEE Transactions on Multimedia.

[41]  Bing-Yu Chen,et al.  A web-based three-dimensional protein retrieval system by matching visual similarity , 2005, Bioinform..

[42]  Nicholas Ayache,et al.  Geometric Means in a Novel Vector Space Structure on Symmetric Positive-Definite Matrices , 2007, SIAM J. Matrix Anal. Appl..

[43]  Robert C Coghill,et al.  Voxel-based morphometry and arterial spin labeling fMRI reveal neuropathic and neuroplastic features of brain processing of itch in end-stage renal disease. , 2014, Journal of neurophysiology.

[44]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[45]  Sven J. Dickinson,et al.  Skeleton based shape matching and retrieval , 2003, 2003 Shape Modeling International..

[46]  Marc Rioux,et al.  Description of shape information for 2-D and 3-D objects , 2000, Signal Process. Image Commun..

[47]  Remco C. Veltkamp,et al.  Polyhedral Model Retrieval Using Weighted Point Sets , 2003, Int. J. Image Graph..

[48]  Chang-Hsing Lee,et al.  A new 3D model retrieval approach based on the elevation descriptor , 2007, Pattern Recognit..

[49]  Weizhi Nie,et al.  Evaluation of local spatial-temporal features for cross-view action recognition , 2016, Neurocomputing.

[50]  Zan Gao,et al.  Multi-view discriminative and structured dictionary learning with group sparsity for human action recognition , 2015, Signal Process..

[51]  David P. Dobkin,et al.  A search engine for 3D models , 2003, TOGS.

[52]  Yue Gao,et al.  3D model comparison using spatial structure circular descriptor , 2010, Pattern Recognit..

[53]  Meng Wang,et al.  3D deep shape descriptor , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[54]  Samy Bengio,et al.  Show and tell: A neural image caption generator , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).