Multi-View Graph Matching for 3D Model Retrieval

3D model retrieval has been widely utilized in numerous domains, such as computer-aided design, digital entertainment, and virtual reality. Recently, many graph-based methods have been proposed to address this task by using multi-view information of 3D models. However, these methods are always constrained by many-to-many graph matching for the similarity measure between pairwise models. In this article, we propose a multi-view graph matching method (MVGM) for 3D model retrieval. The proposed method can decompose the complicated multi-view graph-based similarity measure into multiple single-view graph-based similarity measures and fusion. First, we present the method for single-view graph generation, and we further propose the novel method for the similarity measure in a single-view graph by leveraging both node-wise context and model-wise context. Then, we propose multi-view fusion with diffusion, which can collaboratively integrate multiple single-view similarities w.r.t. different viewpoints and adaptively learn their weights, to compute the multi-view similarity between pairwise models. In this way, the proposed method can avoid the difficulty in the definition and computation of the traditional high-order graph. Moreover, this method is unsupervised and does not require a large-scale 3D dataset for model learning. We conduct evaluations on four popular and challenging datasets. The extensive experiments demonstrate the superiority and effectiveness of the proposed method compared against the state of the art. In particular, this unsupervised method can achieve competitive performances against the most recent supervised and deep learning method.

[1]  Yue Gao,et al.  GVCNN: Group-View Convolutional Neural Networks for 3D Shape Recognition , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[2]  Jin-Hui Zhu,et al.  Simultaneous Recognition and Modeling for Learning 3-D Object Models From Everyday Scenes , 2015, IEEE Transactions on Cybernetics.

[3]  Dapeng Tao,et al.  Deep Multi-View Feature Learning for Person Re-Identification , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  Wenhui Li,et al.  View-wised discriminative ranking for 3D object retrieval , 2018, Multimedia Tools and Applications.

[5]  Sven J. Dickinson,et al.  Skeleton based shape matching and retrieval , 2003, 2003 Shape Modeling International..

[6]  Yue Gao,et al.  Multi-Modal Clique-Graph Matching for View-Based 3D Model Retrieval , 2016, IEEE Transactions on Image Processing.

[7]  Uwe Stilla,et al.  Detection of fallen trees in ALS point clouds of a temperate forest by combining point/primitive-level shape descriptors , 2014 .

[8]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Xuelong Li,et al.  Convex Multiview Semi-Supervised Classification , 2017, IEEE Transactions on Image Processing.

[10]  Yongdong Zhang,et al.  Multi-Level Policy and Reward-Based Deep Reinforcement Learning Framework for Image Captioning , 2020, IEEE Transactions on Multimedia.

[11]  Ricardo da Silva Torres,et al.  Image re-ranking and rank aggregation based on similarity of ranked lists , 2013, Pattern Recognit..

[12]  Ming Ouhyoung,et al.  On Visual Similarity Based 3D Model Retrieval , 2003, Comput. Graph. Forum.

[13]  Zhang Xiong,et al.  A powerful 3D model classification mechanism based on fusing multi-graph , 2015, Neurocomputing.

[14]  Yu-Ting Su,et al.  View-Based 3-D Model Retrieval: A Benchmark , 2018, IEEE Transactions on Cybernetics.

[15]  Francesco G. B. De Natale,et al.  Multimodal Retrieval with Diversification and Relevance Feedback for Tourist Attraction Images , 2017, ACM Trans. Multim. Comput. Commun. Appl..

[16]  Ling Shao,et al.  Deep Nonlinear Metric Learning for 3-D Shape Retrieval , 2018, IEEE Transactions on Cybernetics.

[17]  Jun Wu,et al.  CHCF: A Cloud-Based Heterogeneous Computing Framework for Large-Scale Image Retrieval , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  Robert C Coghill,et al.  Voxel-based morphometry and arterial spin labeling fMRI reveal neuropathic and neuroplastic features of brain processing of itch in end-stage renal disease. , 2014, Journal of neurophysiology.

[19]  Wenhui Li,et al.  Hierarchical Graph Structure Learning for Multi-View 3D Model Retrieval , 2018, IJCAI.

[20]  Yue Gao,et al.  Beyond Text QA: Multimedia Answer Generation by Harvesting Web Information , 2013, IEEE Transactions on Multimedia.

[21]  Yue Gao,et al.  PVNet: A Joint Convolutional Network of Point Cloud and Multi-View for 3D Shape Recognition , 2018, ACM Multimedia.

[22]  Alistair Moffat,et al.  A similarity measure for indefinite rankings , 2010, TOIS.

[23]  Heng Tao Shen,et al.  Exploring Auxiliary Context: Discrete Semantic Transfer Hashing for Scalable Image Retrieval , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[24]  Horst Bischof,et al.  Diffusion Processes for Retrieval Revisited , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Thierry Pun,et al.  Performance evaluation in content-based image retrieval: overview and proposals , 2001, Pattern Recognit. Lett..

[26]  Yue Gao,et al.  3D model retrieval using weighted bipartite graph matching , 2011, Signal Process. Image Commun..

[27]  Ricardo da Silva Torres,et al.  Image Re-ranking and Rank Aggregation Based on Similarity of Ranked Lists , 2011, CAIP.

[28]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[29]  Qionghai Dai,et al.  Cross-Modality Bridging and Knowledge Transferring for Image Understanding , 2019, IEEE Transactions on Multimedia.

[30]  Yuan Yan Tang,et al.  High-Order Distance-Based Multiview Stochastic Learning in Image Classification , 2014, IEEE Transactions on Cybernetics.

[31]  Yue Gao,et al.  3-D Object Retrieval and Recognition With Hypergraph Analysis , 2012, IEEE Transactions on Image Processing.

[32]  Yue Gao,et al.  Camera Constraint-Free View-Based 3-D Object Retrieval , 2012, IEEE Transactions on Image Processing.

[33]  Petros Daras,et al.  A 3D Shape Retrieval Framework Supporting Multimodal Queries , 2010, International Journal of Computer Vision.

[34]  Yoichi Sato,et al.  Uncalibrated photometric stereo based on elevation angle recovery from BRDF symmetry of isotropic materials , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Xuelong Li,et al.  Event-Based Media Enrichment Using an Adaptive Probabilistic Hypergraph Model , 2015, IEEE Transactions on Cybernetics.

[36]  Yuting Su,et al.  Graph-based characteristic view set extraction and matching for 3D model retrieval , 2015, Inf. Sci..

[37]  Miki Haseyama,et al.  [Foreword] Welcome to the Transactions on Media Technology and Applications:The Institute of Image Information and Television Engineers (ITE) has decided to launch a new open access journal, titled "Media Technology and Applications" (MTA). , 2013 .

[38]  Fritz Albregtsen,et al.  Fast and exact computation of Cartesian geometric moments using discrete Green's theorem , 1996, Pattern Recognit..

[39]  Wen Gao,et al.  Mining Compact Bag-of-Patterns for Low Bit Rate Mobile Visual Search , 2014, IEEE Transactions on Image Processing.

[40]  Yue Gao,et al.  View-Based 3D Object Retrieval: Challenges and Approaches , 2014, IEEE MultiMedia.

[41]  Andrea Giachetti,et al.  Scale Space Graph Representation and Kernel Matching for Non Rigid and Textured 3D Shape Retrieval , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  An-An Liu,et al.  3D Object Retrieval Based on Multi-View Latent Variable Model , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[43]  Mohan S. Kankanhalli,et al.  MMALFM , 2018, ACM Trans. Inf. Syst..

[44]  Thomas A. Funkhouser,et al.  The Princeton Shape Benchmark , 2004, Proceedings Shape Modeling Applications, 2004..

[45]  Longin Jan Latecki,et al.  Re-Ranking via Metric Fusion for Object Retrieval and Person Re-Identification , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Meng Wang,et al.  Multi-View Object Retrieval via Multi-Scale Topic Models , 2016, IEEE Transactions on Image Processing.

[47]  Hongxun Yao,et al.  View-based 3D object retrieval via multi-modal graph learning , 2015, Signal Process..

[48]  Deyu Wang,et al.  A Fast 3D Retrieval Algorithm via Class-Statistic and Pair-Constraint Model , 2016, ACM Multimedia.

[49]  Ling Shao,et al.  Zero-Shot Learning Using Synthesised Unseen Visual Data with Diffusion Regularisation , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Zhang Xiong,et al.  A 3D model recognition mechanism based on deep Boltzmann machines , 2015, Neurocomputing.

[51]  Ke Lu,et al.  Heterogeneous Domain Adaptation Through Progressive Alignment , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[52]  Edward K. Wong,et al.  DeepShape: Deep-Learned Shape Descriptor for 3D Shape Retrieval , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Mubarak Shah,et al.  Learning a Multi-concept Video Retrieval Model with Multiple Latent Variables , 2016, 2016 IEEE International Symposium on Multimedia (ISM).

[54]  Caiyan Jia,et al.  Structure-Aware Deep Learning for Product Image Classification , 2019, ACM Trans. Multim. Comput. Commun. Appl..

[55]  Cosimo Rubino,et al.  3D Object Localisation from Multi-View Image Detections , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Ke Lu,et al.  3D model retrieval and classification by semi-supervised learning with content-based similarity , 2014, Inf. Sci..

[57]  Bernard Chazelle,et al.  Matching 3D models with shape distributions , 2001, Proceedings International Conference on Shape Modeling and Applications.

[58]  Deyu Wang,et al.  Group-Pair Convolutional Neural Networks for Multi-View Based 3D Object Retrieval , 2018, AAAI.

[59]  Yongdong Zhang,et al.  STAT: Spatial-Temporal Attention Mechanism for Video Captioning , 2020, IEEE Transactions on Multimedia.

[60]  Mohan S. Kankanhalli,et al.  Hierarchical Clustering Multi-Task Learning for Joint Human Action Grouping and Recognition , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Bernt Schiele,et al.  Analyzing appearance and contour based methods for object categorization , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[62]  Taku Komura,et al.  Topology matching for fully automatic similarity estimation of 3D shapes , 2001, SIGGRAPH.

[63]  Mohamed Daoudi,et al.  A Bayesian 3-D Search Engine Using Adaptive Views Clustering , 2007, IEEE Transactions on Multimedia.

[64]  Yongdong Zhang,et al.  Convolutional Attention Networks for Scene Text Recognition , 2019, ACM Trans. Multim. Comput. Commun. Appl..

[65]  Ivor W. Tsang,et al.  DEFEATnet—A Deep Conventional Image Representation for Image Classification , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[66]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[67]  Hans-Peter Kriegel,et al.  3D Shape Histograms for Similarity Search and Classification in Spatial Databases , 1999, SSD.

[68]  Xuelong Li,et al.  Spectral Multimodal Hashing and Its Application to Multimedia Retrieval , 2016, IEEE Transactions on Cybernetics.

[69]  Qi Tian,et al.  Ensemble Diffusion for Retrieval , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[70]  Liang Zheng,et al.  Re-ranking Person Re-identification with k-Reciprocal Encoding , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[71]  Weizhi Nie,et al.  3D object retrieval based on sparse coding in weak supervision , 2016, J. Vis. Commun. Image Represent..