Deep-based Self-refined Face-top Coordination

Face-top coordination, which exists in most clothes-fitting scenarios, is challenging due to varieties of attributes, implicit correlations, and tradeoffs between general preferences and individual preferences. We present a Deep-Based Self-Refined (DBSR) system to simulate face-top coordination based on intuition evaluation. To this end, we first establish a well-coordinated face-top (WCFT) dataset from fashion databases and communities. Then, we use a jointly trained CNN Deep Canonical Correlation Analysis (DCCA) method to bridge the semantic face-top gap based on the WCFT dataset to deal with general preferences. Subsequently, an irrelevance-based Optimum-path Forest (OPF) method is developed to adapt the results to individual preferences iteratively. Experimental results and user study demonstrate the effectiveness of our method.

[1]  Sudhir Kumar,et al.  c+GAN: Complementary Fashion Item Recommendation , 2019, ArXiv.

[2]  Eytan Ruppin,et al.  Facial Attractiveness: Beauty and the Machine , 2006, Neural Computation.

[3]  Noah Snavely,et al.  GeoStyle: Discovering Fashion Trends and Events , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[4]  Chong-Wah Ngo,et al.  Interpretable Multimodal Retrieval for Fashion Products , 2018, ACM Multimedia.

[5]  Xiao Wu,et al.  Personalized clothing recommendation combining user social circle and fashion style consistency , 2017, Multimedia Tools and Applications.

[6]  Larry S. Davis,et al.  Collaborative Fashion Recommendation: A Functional Tensor Factorization Approach , 2015, ACM Multimedia.

[7]  Santanu Chaudhury,et al.  An Ontology Based Personalized Garment Recommendation System , 2013, 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT).

[8]  Meng Wang,et al.  TransNFCM: Translation-Based Neural Fashion Compatibility Modeling , 2018, AAAI.

[9]  Jie Xu,et al.  SCUT-FBP: A Benchmark Dataset for Facial Beauty Perception , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[10]  Shotaro Akaho,et al.  A kernel method for canonical correlation analysis , 2006, ArXiv.

[11]  Takayuki Okatani,et al.  Toward Explainable Fashion Recommendation , 2019, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[12]  Tomoharu Iwata,et al.  Fashion Coordinates Recommender System Using Photographs from Fashion Magazines , 2011, IJCAI.

[13]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[14]  Jeff A. Bilmes,et al.  On Deep Multi-View Representation Learning , 2015, ICML.

[15]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[16]  Xu Chen,et al.  Aesthetic-based Clothing Recommendation , 2018 .

[17]  Jure Leskovec,et al.  Complete the Look: Scene-Based Complementary Product Recommendation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Matthias Hein,et al.  Variants of RMSProp and Adagrad with Logarithmic Regret Bounds , 2017, ICML.

[19]  Changsheng Xu,et al.  Hi, magic closet, tell me what to wear! , 2012, ACM Multimedia.

[20]  Wen-Huang Cheng,et al.  What Dress Fits Me Best?: Fashion Recommendation on the Clothing Style for Personal Body Shape , 2018, ACM Multimedia.

[21]  Serge J. Belongie,et al.  Learning Visual Clothing Style with Heterogeneous Dyadic Co-Occurrences , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Shuicheng Yan,et al.  "Wow! You Are So Beautiful Today!" , 2014, ACM Trans. Multim. Comput. Commun. Appl..

[23]  Alexander C. Berg,et al.  Hipster Wars: Discovering Elements of Fashion Styles , 2014, ECCV.

[24]  Francesc Moreno-Noguer,et al.  Neuroaesthetics in fashion: Modeling the perception of fashionability , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Karen Livescu,et al.  Multi-view Recurrent Neural Acoustic Word Embeddings , 2016, ICLR.

[26]  Jeff A. Bilmes,et al.  Deep Canonical Correlation Analysis , 2013, ICML.

[27]  Jianfei Cai,et al.  M2E-Try On Net: Fashion from Model to Everyone , 2018, ACM Multimedia.

[28]  Mario Fritz,et al.  Fashion Is Taking Shape: Understanding Clothing Preference Based on Body Shape From Online Sources , 2018, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[29]  Yutaka Satoh,et al.  Changing Fashion Cultures , 2017, ArXiv.

[30]  Jianguo Wang,et al.  Sherlock: Sparse Hierarchical Embeddings for Visually-Aware One-Class Collaborative Filtering , 2016, IJCAI.

[31]  Wei Yang,et al.  Retrieval of clothing images based on relevance feedback with focus on collar designs , 2016, The Visual Computer.

[32]  Yang Yang,et al.  Adversarial Cross-Modal Retrieval , 2017, ACM Multimedia.

[33]  Yejun Liu,et al.  Towards Better Understanding the Clothing Fashion Styles: A Multimodal Deep Learning Approach , 2017, AAAI.

[34]  Lei Chen,et al.  Deep Cross-Modal Correlation Learning for Audio and Lyrics in Music Retrieval , 2017, ACM Trans. Multim. Comput. Commun. Appl..

[35]  Zhaochun Ren,et al.  Explainable Outfit Recommendation with Joint Outfit Matching and Comment Generation , 2018, IEEE Transactions on Knowledge and Data Engineering.

[36]  Kristen Grauman,et al.  Learning the Latent “Look”: Unsupervised Discovery of a Style-Coherent Embedding from Fashion Images , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[37]  Dezhong Peng,et al.  Deep Supervised Cross-Modal Retrieval , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Julian J. McAuley,et al.  Learning Compatibility Across Categories for Heterogeneous Item Recommendation , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[39]  Le Wu,et al.  Explainable Fashion Recommendation: A Semantic Attribute Region Guided Approach , 2019, IJCAI.

[40]  Alexandre X. Falcão,et al.  Active learning paradigms for CBIR systems based on optimum-path forest classification , 2011, Pattern Recognit..

[41]  Yann LeCun,et al.  Learning a similarity metric discriminatively, with application to face verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[42]  Bingbing Ni,et al.  Sense beauty via face, dressing, and/or voice , 2012, ACM Multimedia.

[43]  Tat-Seng Chua,et al.  Who, Where, and What to Wear?: Extracting Fashion Knowledge from Social Media , 2019, ACM Multimedia.

[44]  Anton van den Hengel,et al.  Image-Based Recommendations on Styles and Substitutes , 2015, SIGIR.

[45]  Andrea Bottino,et al.  The Analysis of Facial Beauty: An Emerging Area of Research in Pattern Analysis , 2010, ICIAR.

[46]  A. Cellerino,et al.  Shape analysis of female facial attractiveness , 2006, Vision Research.

[47]  Xin Guo,et al.  POG: Personalized Outfit Generation for Fashion Recommendation at Alibaba iFashion , 2019, KDD.

[48]  Ranjitha Kumar,et al.  The Elements of Fashion Style , 2016, UIST.

[49]  Kristen Grauman,et al.  Creating Capsule Wardrobes from Fashion Images , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[50]  Demetri Terzopoulos,et al.  DressUp! , 2012, ACM Trans. Graph..

[51]  Yongfeng Zhang,et al.  Personalized Fashion Recommendation with Visual Explanations based on Multimodal Attention Network: Towards Visually Explainable Recommendation , 2019, SIGIR.

[52]  Jiebo Luo,et al.  Mining Fashion Outfit Composition Using an End-to-End Deep Learning Approach on Set Data , 2016, IEEE Transactions on Multimedia.

[53]  Hui Xiong,et al.  Deep Cross Networks with Aesthetic Preference for Cross-domain Recommendation , 2019, ArXiv.

[54]  Noah Snavely,et al.  StreetStyle: Exploring world-wide clothing styles from millions of photos , 2017, ArXiv.

[55]  Rainer Stiefelhagen,et al.  Fashion Forward: Forecasting Visual Style in Fashion , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[56]  Robinson Piramuthu,et al.  Large scale visual recommendations from street fashion images , 2014, KDD.

[57]  Zhi Lu,et al.  Learning Binary Code for Personalized Fashion Recommendation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Hong-Han Shuai,et al.  FashionOn: Semantic-guided Image-based Virtual Try-on with Detailed Human and Clothing Information , 2019, ACM Multimedia.

[59]  Julian J. McAuley,et al.  VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback , 2015, AAAI.

[60]  G. Tiberghien,et al.  Symmetry, averageness, and feature size in the facial attractiveness of women. , 2004, Acta psychologica.

[61]  Tat-Seng Chua,et al.  Interpretable Fashion Matching with Rich Attributes , 2019, SIGIR.

[62]  Yang Hu,et al.  FashionNet: Personalized Outfit Recommendation with Deep Neural Network , 2018, ArXiv.

[63]  Liqing Zhang,et al.  Fashion Sensitive Clothing Recommendation Using Hierarchical Collocation Model , 2018, ACM Multimedia.

[64]  Wei Yang,et al.  Hairstyle Suggestion Using Statistical Learning , 2012, MMM.

[65]  Luis E. Ortiz,et al.  Chic or Social: Visual Popularity Analysis in Online Fashion Networks , 2014, ACM Multimedia.

[66]  Shuicheng Yan,et al.  "Wow! you are so beautiful today!" , 2013, MM '13.

[67]  Hiroshi Ishikawa,et al.  What Makes a Style: Experimental Analysis of Fashion Prediction , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).