Pedestrian Color Naming via Convolutional Neural Network

Color serves as an important cue for many computer vision tasks. Nevertheless, obtaining accurate color description from images is non-trivial due to varying illumination conditions, view angles, and surface reflectance. This is especially true for the challenging problem of pedestrian description in public spaces. We made two contributions in this study: (1) We contribute a large-scale pedestrian color naming dataset with 14,213 hand-labeled images. (2) We address the problem of assigning consistent color name to regions of single object’s surface. We propose an end-to-end, pixel-to-pixel convolutional neural network (CNN) for pedestrian color naming. We demonstrate that our Pedestrian Color Naming CNN (PCN-CNN) is superior over existing approaches in providing consistent color names on real-world pedestrian images. In addition, we show the effectiveness of color descriptor extracted from PCN-CNN in complementing existing descriptors for the task of person re-identification. Moreover, we discuss a novel application to retrieve outfit matching and fashion (which could be difficult to be described by keywords) with just a user-provided color sketch.

[1]  Shuicheng Yan,et al.  Fashion Parsing With Weak Color-Category Labels , 2014, IEEE Transactions on Multimedia.

[2]  Shengcai Liao,et al.  Person re-identification by Local Maximal Occurrence representation and metric learning , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Jonathan T. Barron,et al.  Convolutional Color Constancy , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[4]  Michael S. Brown,et al.  Effective learning-based illuminant estimation using simple features , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Xiaoxiao Li,et al.  Semantic Image Segmentation via Deep Parsing Network , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[6]  Joost van de Weijer,et al.  Describing Reflectances for Color Segmentation Robust to Shadows, Highlights, and Textures , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Sameh Khamis,et al.  Person re-identification using semantic color names and RankBoost , 2013, 2013 IEEE Workshop on Applications of Computer Vision (WACV).

[8]  Cordelia Schmid,et al.  Learning Color Names for Real-World Applications , 2009, IEEE Transactions on Image Processing.

[9]  Gernot A. Fink,et al.  Web-Based Learning of Naturalized Color Models for Human-Machine Interaction , 2010, 2010 International Conference on Digital Image Computing: Techniques and Applications.

[10]  Alexei A. Efros,et al.  Detecting Ground Shadows in Outdoor Consumer Photographs , 2010, ECCV.

[11]  Ehud Rivlin,et al.  Color Invariants for Person Reidentification , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[13]  Chunxiao Liu,et al.  Person Re-identification: What Features Are Important? , 2012, ECCV Workshops.

[14]  Katsushi Ikeuchi,et al.  Separating Reflection Components of Textured Surfaces Using a Single Image , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Jian-Huang Lai,et al.  Mirror Representation for Modeling View-Specific Transform in Person Re-Identification , 2015, IJCAI.

[16]  Shaogang Gong,et al.  Person Re-Identification , 2014 .

[17]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[18]  María Vanrell,et al.  Names and shades of color for intrinsic image estimation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Nanning Zheng,et al.  Illumination Robust Color Naming via Label Propagation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[20]  Shaogang Gong,et al.  Person re-identification by probabilistic relative distance comparison , 2011, CVPR 2011.

[21]  Derek Hoiem,et al.  Single-image shadow detection and removal using paired regions , 2011, CVPR 2011.

[22]  Shengcai Liao,et al.  Salient Color Names for Person Re-identification , 2014, ECCV.

[23]  William T. Freeman,et al.  Learning low-level vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[24]  Vittorio Murino,et al.  Symmetry-driven accumulation of local features for human characterization and re-identification , 2013, Comput. Vis. Image Underst..

[25]  Horst Bischof,et al.  Mahalanobis Distance Learning for Person Re-identification , 2014, Person Re-Identification.

[26]  Stephen Lin,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Aleksandra Mojsilovic,et al.  A computational model for color naming and describing color composition of images , 2005, IEEE Transactions on Image Processing.

[28]  Qi Tian,et al.  Scalable Person Re-identification: A Benchmark , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[29]  Feng Liu,et al.  Sketch Me That Shoe , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Shaogang Gong,et al.  Person Re-identification by Attributes , 2012, BMVC.

[31]  Ming-Hsuan Yang,et al.  An Ensemble Color Model for Human Re-identification , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[32]  Hai Tao,et al.  Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features , 2008, ECCV.

[33]  P. Kay,et al.  Basic Color Terms: Their Universality and Evolution , 1973 .

[34]  M. Vanrell,et al.  Parametric fuzzy sets for automatic color naming. , 2008, Journal of the Optical Society of America. A, Optics, image science, and vision.

[35]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Nanning Zheng,et al.  Similarity Learning with Spatial Constraints for Person Re-identification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Xiaogang Wang,et al.  Person Re-identification by Salience Matching , 2013, 2013 IEEE International Conference on Computer Vision.

[38]  Xiaogang Wang,et al.  Pedestrian Parsing via Deep Decompositional Network , 2013, 2013 IEEE International Conference on Computer Vision.

[39]  David A. Forsyth,et al.  Finding glass , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[40]  Horst Bischof,et al.  Relaxed Pairwise Learned Metric for Person Re-identification , 2012, ECCV.

[41]  Claudio Cusano,et al.  Single and Multiple Illuminant Estimation Using Convolutional Neural Networks , 2015, IEEE Transactions on Image Processing.

[42]  Xiaogang Wang,et al.  DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Hai Tao,et al.  Evaluating Appearance Models for Recognition, Reacquisition, and Tracking , 2007 .

[44]  Alessandro Perina,et al.  Person re-identification by symmetry-driven accumulation of local features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.