FashionAI: A Hierarchical Dataset for Fashion Understanding

Fine-grained attribute recognition is critical for fashion understanding, yet is missing in existing professional and comprehensive fashion datasets. In this paper, we present a large scale attribute dataset with manual annotation in high quality. To this end, complex fashion knowledge is disassembled into mutually exclusive concepts and form a hierarchical structure to describe the cognitive process. Such well-structured knowledge is reflected by dataset in terms of its clear definition and precise annotation. The problems which are common in the process of annotation, including structured noise, occlusion, uncertain problems, and attribute inconsistency, are well addressed instead of merely discarding those bad data. Further, we propose an iterative process of building a dataset with practical usefulness. With 24 key points, 245 labels that cover 6 categories of women's clothing, and a total of 41 subcategories, the creation of our dataset drew upon a large amount of crowd staff engagement. Extensive experiments quantitatively and qualitatively demonstrate its effectiveness.

[1]  Liang Lin,et al.  Clothing Co-parsing by Joint Image Segmentation and Labeling , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Luis E. Ortiz,et al.  Parsing clothing in fashion photographs , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Larry S. Davis,et al.  VITON: An Image-Based Virtual Try-on Network , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[4]  Ruimao Zhang,et al.  DeepFashion2: A Versatile Benchmark for Detection, Pose Estimation, Segmentation and Re-Identification of Clothing Images , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Tamara L. Berg,et al.  Paper Doll Parsing: Retrieving Similar Styles to Parse Clothing Items , 2013, 2013 IEEE International Conference on Computer Vision.

[6]  Zhipeng Wu,et al.  Looking at Outfit to Parse Clothing , 2017, ArXiv.

[7]  Luc Van Gool,et al.  Apparel Classification with Style , 2012, ACCV.

[8]  Takayuki Okatani,et al.  Mix and Match: Joint Model for Clothing and Attribute Recognition , 2015, BMVC.

[9]  Zhaochun Ren,et al.  Explainable Outfit Recommendation with Joint Outfit Matching and Comment Generation , 2018, IEEE Transactions on Knowledge and Data Engineering.

[10]  Song-Chun Zhu,et al.  Attentive Fashion Grammar Network for Fashion Landmark Detection and Clothing Category Classification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Jian Dong,et al.  Deep domain adaptation for describing people based on fine-grained clothing attributes , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Changsheng Xu,et al.  Hi, magic closet, tell me what to wear! , 2012, ACM Multimedia.

[13]  Q. Liu,et al.  FashionGAN: Display your fashion design using Conditional Generative Adversarial Nets , 2018, Comput. Graph. Forum.

[14]  Jo Yew Tham,et al.  Learning Attribute Representations with Localization for Flexible Fashion Search , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15]  Zunlei Feng,et al.  Interpretable Partitioned Embedding for Customized Multi-item Fashion Outfit Composition , 2018, ICMR.

[16]  Takayuki Okatani,et al.  Toward Explainable Fashion Recommendation , 2019, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[17]  Alexander C. Berg,et al.  Hipster Wars: Discovering Elements of Fashion Styles , 2014, ECCV.

[18]  Tomoharu Iwata,et al.  Fashion Coordinates Recommender System Using Photographs from Fashion Magazines , 2011, IJCAI.

[19]  Svetlana Lazebnik,et al.  Where to Buy It: Matching Street Clothing Photos in Online Shops , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[20]  Xu Chen,et al.  Visually Explainable Recommendation , 2018, ArXiv.

[21]  Winston Hsu,et al.  Netizen-Style Commenting on Fashion Photos: Dataset and Diversity Measures , 2018, WWW.

[22]  Noah Snavely,et al.  StreetStyle: Exploring world-wide clothing styles from millions of photos , 2017, ArXiv.

[23]  Yu-Gang Jiang,et al.  Learning Fashion Compatibility with Bidirectional LSTMs , 2017, ACM Multimedia.

[24]  Robinson Piramuthu,et al.  Style Finder: Fine-Grained Clothing Style Detection and Retrieval , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[25]  Sanja Fidler,et al.  Be Your Own Prada: Fashion Synthesis with Structural Coherence , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[26]  Shuicheng Yan,et al.  Fashion Parsing With Weak Color-Category Labels , 2014, IEEE Transactions on Multimedia.

[27]  Min Xu,et al.  Efficient Clothing Retrieval with Semantic-Preserving Visual Phrases , 2012, ACCV.

[28]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Hiroshi Ishikawa,et al.  What Makes a Style: Experimental Analysis of Fashion Prediction , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[30]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Qiang Chen,et al.  Cross-Domain Image Retrieval with a Dual Attribute-Aware Ranking Network , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[32]  Wei Liu,et al.  Neural Compatibility Modeling with Attentive Knowledge Distillation , 2018, SIGIR.

[33]  Changsheng Xu,et al.  Street-to-shop: Cross-scenario clothing retrieval via parts alignment and auxiliary set , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Francesc Moreno-Noguer,et al.  Neuroaesthetics in fashion: Modeling the perception of fashionability , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Hanqing Lu,et al.  Street-to-shop: Cross-scenario clothing retrieval via parts alignment and auxiliary set , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Xiaogang Wang,et al.  DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Kristen Grauman,et al.  Learning the Latent “Look”: Unsupervised Discovery of a Style-Coherent Embedding from Fashion Images , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[38]  Ying Zhang,et al.  Fashion-Gen: The Generative Fashion Dataset and Challenge , 2018, ArXiv.