Similarity metric learning for sketch-based 3D object retrieval

Sketch-Based 3D Object Retrieval (SB3DOR) algorithms retrieve 3D models similar to hand-drawn sketch queries. It is one of the most effective modalities to query 3D models by their shape. However, comparison of a sketch, which is a 2D image, with a 3D model is not straightforward. Most of the SB3DOR algorithms compare a sketch image with images of 3D models rendered from multiple viewpoints. However, retrieval accuracies of state-of-the-art SB3DOR algorithms are still not satisfactory, due, in part, to stylistic variation, semantic influence, abstraction, and drawing error found in sketches. To improve retrieval accuracy of SB3DOR systems, we propose a manifold-based similarity metric learning algorithm that relates two kinds of features, that are, of sketches and 3D models. Features in a high dimensional space often lie on a lower dimensional subspace, or manifold. Feature similarity may be computed more accurately on the manifold than in the original high dimensional space. Our Cross-Domain Manifold Ranking (CDMR) algorithm tries to keep the two distinct feature manifolds, one for sketch features and the other for 3D model features, intact. These two manifolds are interlinked by using inter-feature similarity to form a Cross-Domain Manifold (CDM). If available, semantic labels may also be used in forming the CDM. Relevance diffusion is used to compute similarities between a sketch and 3D models in a database. Experimental evaluation showed that the CDMR algorithm produces higher retrieval accuracy than the algorithms we compared against.