Semantic Embedding for Sketch-Based 3D Shape Retrieval

The main challenge for sketch-based 3D shape retrieval lies with the large domain gap between 2D sketch and 3D shape. Most existing works attempt to overcome the domain gap by learning a joint feature embedding space to align the two domains. In this work we argue that the large domain gap cannot be effectively bridged in a shared feature space. Instead, we propose to align them in their common class label space. To this end, a novel deep cross-domain semantic embedding model is proposed. Extensive experiments are carried out on two large benchmarking datasets, SHREC’13 and SHREC’14. The results show that the proposed model drastically improves over the state-of-the-art alternatives.

[1]  Honggang Zhang,et al.  Sketch-based image retrieval via Siamese convolutional neural network , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[2]  Xiaogang Wang,et al.  Learning Deep Feature Representations with Domain Guided Dropout for Person Re-identification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Rui Hu,et al.  A performance evaluation of gradient field HOG descriptor for sketch based image retrieval , 2013, Comput. Vis. Image Underst..

[5]  C. Lawrence Zitnick,et al.  Bringing Semantics into Focus Using Visual Abstraction , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Tong Lu,et al.  A new recognition model for electronic architectural drawings , 2005, Comput. Aided Des..

[7]  Bo Li,et al.  SHREC'13 Track: Large Scale Sketch-Based 3D Shape Retrieval , 2013, 3DOR@Eurographics.

[8]  Feng Liu,et al.  Sketch Me That Shoe , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Chi-Ho Chan,et al.  Evaluation of face recognition system in heterogeneous environments (visible vs NIR) , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[10]  Anil K. Jain,et al.  Heterogeneous Face Recognition: Matching NIR to Visible Light Images , 2010, 2010 20th International Conference on Pattern Recognition.

[11]  Shaogang Gong,et al.  Person Re-Identification by Deep Joint Learning of Multi-Loss Classification , 2017, IJCAI.

[12]  Marc Alexa,et al.  Sketch-based shape retrieval , 2012, ACM Trans. Graph..

[13]  Tao Xiang,et al.  Multi-level Factorisation Net for Person Re-identification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Shaogang Gong,et al.  Free-hand sketch recognition by multi-kernel feature learning , 2015, Comput. Vis. Image Underst..

[16]  Fang Wang,et al.  Sketch-based 3D shape retrieval using Convolutional Neural Networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Tao Xiang,et al.  Sketch-a-Net: A Deep Neural Network that Beats Humans , 2017, International Journal of Computer Vision.

[18]  Marc Alexa,et al.  How do humans sketch objects? , 2012, ACM Trans. Graph..

[19]  Tinne Tuytelaars,et al.  Sketch classification and classification-driven analysis using Fisher vectors , 2014, ACM Trans. Graph..

[20]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Petros Daras,et al.  A 3D Shape Retrieval Framework Supporting Multimodal Queries , 2010, International Journal of Computer Vision.

[22]  Geoffrey E. Hinton,et al.  Regularizing Neural Networks by Penalizing Confident Output Distributions , 2017, ICLR.

[23]  Bo Li,et al.  A comparison of methods for sketch-based 3D shape retrieval , 2014, Comput. Vis. Image Underst..

[24]  Ravi Kiran Sarvadevabhatla,et al.  Freehand Sketch Recognition Using Deep Features , 2015, ArXiv.

[25]  Tao Xiang,et al.  Deep Multi-task Attribute-driven Ranking for Fine-grained Sketch-based Image Retrieval , 2016, BMVC.

[26]  Lucas Beyer,et al.  In Defense of the Triplet Loss for Person Re-Identification , 2017, ArXiv.

[27]  Leonidas J. Guibas,et al.  Volumetric and Multi-view CNNs for Object Classification on 3D Data , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  M. Eitz,et al.  Sketch-based 3 D shape retrieval , 2010 .

[29]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[30]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[31]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[32]  Yi Fang,et al.  Learning Barycentric Representations of 3D Shapes for Sketch-Based 3D Shape Retrieval , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Thomas A. Funkhouser,et al.  The Princeton Shape Benchmark , 2004, Proceedings Shape Modeling Applications, 2004..

[34]  Neil A. Dodgson,et al.  Shape2Vec: semantic-based descriptors for 3D shapes, sketches and images , 2016, ACM Trans. Graph..

[35]  Marc Alexa,et al.  Sketch-Based Image Retrieval: Benchmark and Bag-of-Features Descriptors , 2011, IEEE Transactions on Visualization and Computer Graphics.

[36]  Yi Fang,et al.  Deep Correlated Metric Learning for Sketch-based 3D Shape Retrieval , 2017, AAAI.

[37]  Shengcai Liao,et al.  Heterogeneous Face Recognition from Local Structures of Normalized Appearance , 2009, ICB.

[38]  Bo Li,et al.  Extended Large Scale Sketch-Based 3D Shape Retrieval , 2014, 3DOR@Eurographics.

[39]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[40]  AxenopoulosApostolos,et al.  A 3D Shape Retrieval Framework Supporting Multimodal Queries , 2010 .

[41]  Shaogang Gong,et al.  Highly Efficient Regression for Scalable Person Re-Identification , 2016, BMVC.