Which shirt for my first date? Towards a flexible attribute-based fashion query system

Abstract In this paper, we present a query & retrieval system specially designed for upper-body clothing products such as Shirts, Jackets etc. To facilitate precise learning, each clothing product image is divided into its constituent parts (e.g. torso and sleeves) to enable the learning of the respective attributes of the parts. The regions associated with the constituent clothing parts are first identified by an algorithm which exploits the structure of the upper-body clothing items. Our proposed “Part Guided” Convolutional Neural Network (CNN) is then “steered” towards different regions according to the attributes and expedite the training. In order to extract compact color information and enable flexible query formulation in a richer space, a Markov Random Field based color encoding method is adopted. A detailed dataset called “Shirt Attributes Dataset” was constructed to evaluate the performance of the proposed system. The proposed system allows more specific query formulations and achieves better results compared to different baselines in terms of classification and retrieval accuracy.