Deep model style: Cross-class style compatibility for 3D furniture within a scene

Harmonizing the style of all the furniture placed within a constrained space/scene has been regarded as one of the most important tasks in interior design. Most previous style analysis works measure the style similarity or compatibility of the objects based on predefined geometric features extracted from 3D models. However, “style” is a high-level semantic concept, which is difficult to be described explicitly by handcrafted geometric features. Deep neural network has been claimed to have more powerful ability to mimic the perception of human visual cortex. Therefore, in this work we utilize Triplet Convolutional Neural Network (Triplet CNN) to analyze style compatibility between 3D furniture models of different classes (e.g., a table and a lamp). It should be noted that analyzing the style compatibility between two or more furniture of different classes is quite difficult, as the given furniture may have distinctive structures or geometric elements. We conducted experiments based on a collected dataset containing 420 textured 3D furniture models. A group of raters were recruited from Amazon Mechanical Turk (AMT) to evaluate the comparative suitability of paired models within the dataset. The experimental results reveal that the proposed furniture style compatibility method based on deep learning is better than the state-of-the-art method and can be used for furniture recommendation.

[1]  Ricardo Imbert,et al.  Comparative analysis of shape descriptors for 3D objects , 2017, Multimedia Tools and Applications.

[2]  Abhinav Gupta,et al.  Marr Revisited: 2D-3D Alignment via Surface Normal Prediction , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Leonidas J. Guibas,et al.  ShapeNet: An Information-Rich 3D Model Repository , 2015, ArXiv.

[4]  Nir Ailon,et al.  Deep Metric Learning Using Triplet Network , 2014, SIMBAD.

[5]  Leif Kobbelt,et al.  Identifying Style of 3D Shapes using Deep Metric Learning , 2016, Comput. Graph. Forum.

[6]  Serge J. Belongie,et al.  Cost-Effective HITs for Relative Similarity Comparisons , 2014, HCOMP.

[7]  Edward K. Wong,et al.  DeepShape: Deep-Learned Shape Descriptor for 3D Shape Retrieval , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Babak Saleh,et al.  Learning style similarity for searching infographics , 2015, Graphics Interface.

[9]  Brian Kulis,et al.  Metric Learning: A Survey , 2013, Found. Trends Mach. Learn..

[10]  Krystian Mikolajczyk,et al.  PN-Net: Conjoined Triple Deep Network for Learning Local Image Descriptors , 2016, ArXiv.

[11]  Diego Gutierrez,et al.  A similarity measure for illustration style , 2014, ACM Trans. Graph..

[12]  Aaron Hertzmann,et al.  Exploratory font selection using crowdsourced attributes , 2014, ACM Trans. Graph..

[13]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[14]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[15]  Wilmot Li,et al.  Style compatibility for 3D furniture models , 2015, ACM Trans. Graph..

[16]  Diego Gutierrez,et al.  Style-based exploration of illustration datasets , 2017, Multimedia Tools and Applications.

[17]  Manfred Lau,et al.  Improving Style Similarity Metrics of 3D Shapes , 2015, Graphics Interface.

[18]  Kavita Bala,et al.  Learning visual similarity for product design with convolutional neural networks , 2015, ACM Trans. Graph..

[19]  Vincent Lepetit,et al.  Learning descriptors for object recognition and 3D pose estimation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Alla Sheffer,et al.  Elements of style , 2015, ACM Trans. Graph..

[21]  Leonidas J. Guibas,et al.  Fine-grained semi-supervised labeling of large shape collections , 2013, ACM Trans. Graph..

[22]  D. Cohen-Or,et al.  Style-content separation by anisotropic part scales , 2010, ACM Trans. Graph..