Appearance Descriptors for Person Re-identification: a Comprehensive Review

In video-surveillance, person re- identication is the task of recognising whether an individual has already been observed over a network of cameras. Typically, this is achieved by exploiting the clothing appearance, as classical biometric traits like the face are impractical in real-world video surveil- lance scenarios. Clothing appearance is represented by means of low-level local and/or global features of the image, usually extracted according to some part- based body model to treat dierent body parts (e.g. torso and legs) independently. This paper provides a comprehensive review of current approaches to build appearance descriptors for person re-identication. The most relevant techniques are described in detail, and categorised according to the body models and features used. The aim of this work is to provide a structured body of knowledge and a starting point for researchers willing to conduct novel investigations on this challenging topic.

[1]  G. S. Daniels.  THE "AVERAGE MAN" ? , 1952 .

[2]  John A. Roebuck,et al.  Engineering Anthropometry Methods , 1975 .

[3]  G. Buchsbaum.  A spatial processor model for object colour perception , 1980 .

[4]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[6]  Yali Amit,et al.  Graphical Templates for Model Registration , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  J. C. BurgesChristopher.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998 .

[8]  S. Stevenage,et al.  Visual analysis of gait as a cue to identity , 1999 .

[9]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.

[10]  Ioannis A. Kakadiaris,et al.  Estimating Anthropometry and Pose from a Single Uncalibrated Image , 2001, Comput. Vis. Image Underst..

[11]  B. S. Manjunath,et al.  Color and texture descriptors , 2001, IEEE Trans. Circuits Syst. Video Technol..

[12]  B. S. Manjunath,et al.  An efficient color representation for image retrieval , 2001, IEEE Trans. Image Process..

[13]  Michael Isard,et al.  BraMBLe: a Bayesian multiple-blob tracker , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[14]  Cordelia Schmid,et al.  Constructing models for content-based image retrieval , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[15]  Thomas Sikora,et al.  The MPEG-7 visual standard for content description-an overview , 2001, IEEE Trans. Circuits Syst. Video Technol..

[16]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[17]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Yücel Altunbasak,et al.  Eigenface-domain super-resolution for face recognition , 2003, IEEE Trans. Image Process..

[19]  Afzal Godil,et al.  Human identification from body shape , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[20]  J. Tasic,et al.  Colour spaces: perceptual, historical and applicational background , 2003, The IEEE Region 8 EUROCON 2003. Computer as a Tool..

[21]  Yuan-Fang Wang,et al.  Real-time multiperson tracking in video surveillance , 2003, Fourth International Conference on Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia. Proceedings of the 2003 Joint.

[22]  Arun Ross,et al.  Multimodal biometrics: An overview , 2004, 2004 12th European Signal Processing Conference.

[23]  Daniel P. Huttenlocher,et al.  Pictorial Structures for Object Recognition , 2004, International Journal of Computer Vision.

[24]  M. Hahnel,et al.  Color and texture features for person recognition , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[25]  Mubarak Shah,et al.  Appearance modeling for tracking in multiple non-overlapping cameras , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[26]  Gerald Schaefer,et al.  Illuminant and device invariant colour using histogram equalisation , 2005, Pattern Recognit..

[27]  Shaogang Gong,et al.  Multi-modal tensor face for simultaneous super-resolution and recognition , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[28]  Robert P. W. Duin,et al.  The Dissimilarity Representation for Pattern Recognition - Foundations and Applications , 2005, Series in Machine Perception and Artificial Intelligence.

[29]  Pedro F. Felzenszwalb.  Representation and detection of deformable shapes , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[30]  Massimo Piccardi,et al.  Track matching over disjoint camera views based on an incremental major color spectrum histogram , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[31]  Massimo Piccardi,et al.  Height measurement as a session-based biometric for people matching across disjoint camera views , 2005 .

[32]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Michael J. Black,et al.  Predicting 3D People from 2D Pictures , 2006, AMDO.

[34]  Richard I. Hartley,et al.  Person Reidentification Using Spatiotemporal Appearance , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[35]  Roberto Cipolla,et al.  Face recognition from video , 2006 .

[36]  Bir Bhanu,et al.  Individual recognition using gait energy image , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Deva Ramanan,et al.  Learning to parse images of articulated bodies , 2006, NIPS.

[38]  Hua Li,et al.  3D gait recognition using multiple cameras , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[39]  M. Shah,et al.  Object tracking: A survey , 2006, CSUR.

[40]  Per-Erik Forssén,et al.  Maximally Stable Colour Regions for Recognition and Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Hai Tao,et al.  Evaluating Appearance Models for Recognition, Reacquisition, and Tracking , 2007 .

[42]  Fabien Moutarde,et al.  Person re-identification in multi-camera system by signature based on interest point descriptors collected on short video sequences , 2008, 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras.

[43]  David A. McAllester,et al.  A discriminatively trained, multiscale, deformable part model , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Omar Hamdoun,et al.  Interest points harvesting in video sequences for efficient person identification , 2008 .

[45]  Y. Yacoob,et al.  Statistical Estimation of Human Anthropometry from a Single Uncalibrated Image , 2008 .

[46]  Thierry Bouwmans,et al.  Background Modeling using Mixture of Gaussians for Foreground Detection - A Survey , 2008 .

[47]  Yaser Yacoob,et al.  Statistical body height estimation from a single image , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[48]  Wei-Han Chang,et al.  A fast MPEG-7 dominant color extraction with new similarity measure for image retrieval , 2008, J. Vis. Commun. Image Represent..

[49]  Pablo H. Hennings-Yeomans,et al.  Simultaneous super-resolution and feature extraction for recognition of low-resolution faces , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Stefan Roth,et al.  People-tracking-by-detection and people-detection-by-tracking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[52]  Hai Tao,et al.  Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features , 2008, ECCV.

[53]  Louahdi Khoudour,et al.  Video Sequences Association for People Re-identification across Multiple Non-overlapping Cameras , 2009, ICIAP.

[54]  Shaogang Gong,et al.  Associating Groups of People , 2009, BMVC.

[55]  I. O. D. Oliveira,et al.  Object Reidentification in Multiple Cameras System , 2009 .

[56]  Bernt Schiele,et al.  Pictorial structures revisited: People detection and articulated pose estimation , 2009, CVPR.

[57]  Shiguang Shan,et al.  Coupled Metric Learning for Face Recognition with Degraded Images , 2009, ACML.

[58]  Tsuhan Chen,et al.  Jointly estimating demographics and height with a calibrated camera , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[59]  Slawomir Bak,et al.  Person Re-identification Using Haar-based and DCD-based Signature , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[60]  Koen E. A. van de Sande,et al.  Evaluating Color Descriptors for Object and Scene Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Rama Chellappa,et al.  Evaluation of state-of-the-art algorithms for remote face recognition , 2010, 2010 IEEE International Conference on Image Processing.

[62]  Louahdi Khoudour,et al.  People re-identification by spectral classification of silhouettes , 2010, Signal Process..

[63]  Yang Wang,et al.  Appearance-Based Re-identification of People in Video , 2010, 2010 International Conference on Digital Image Computing: Techniques and Applications.

[64]  Angela D'angelo,et al.  A statistical approach to culture colors distribution in video sensors , 2010 .

[65]  Alessandro Perina,et al.  Multiple-Shot Person Re-identification by HPE Signature , 2010, 2010 20th International Conference on Pattern Recognition.

[66]  Shaogang Gong,et al.  Person Re-Identification by Support Vector Ranking , 2010, BMVC.

[67]  Phil Sallee,et al.  Training and feature-reduction techniques for human identification using anthropometry , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[68]  Matti Pietikäinen,et al.  Person Re-identification Based on Global Color Context , 2010, ACCV Workshops.

[69]  Slawomir Bak,et al.  Person Re-identification Using Spatial Covariance Regions of Human Body Parts , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[70]  Jong-Soo Choi,et al.  A single-view based framework for robust estimation of heights and positions of moving people , 2010 .

[71]  Shiguang Shan,et al.  Low-Resolution Face Recognition via Coupled Locality Preserving Mappings , 2010, IEEE Signal Processing Letters.

[72]  Patrick J. Flynn,et al.  Multidimensional Scaling for Matching Low-Resolution Face Images , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[73]  Junxia Gu,et al.  Action and Gait Recognition From Recovered 3-D Human Joints , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[74]  Alessandro Perina,et al.  Person re-identification by symmetry-driven accumulation of local features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[75]  Shishir K. Shah,et al.  Multiple person re-identification using part based spatio-temporal color appearance model , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[76]  Vittorio Murino,et al.  Custom Pictorial Structures for Re-identification , 2011, BMVC.

[77]  Masayuki Mukunoki,et al.  Optimizing Mean Reciprocal Rank for person re-identification , 2011, 2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[78]  Shaogang Gong,et al.  Person re-identification by probabilistic relative distance comparison , 2011, CVPR 2011.

[79]  Rama Chellappa,et al.  Synthesis-based recognition of low resolution faces , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[80]  Fabio Roli,et al.  A Multiple Component Matching Framework for Person Re-identification , 2011, ICIAP.

[81]  Peter H. Tu,et al.  Appearance-based person reidentification in camera networks: problem overview and current approaches , 2011, J. Ambient Intell. Humaniz. Comput..

[82]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[83]  Michael Arens,et al.  Person re-identification in multi-camera networks , 2011, CVPR 2011 WORKSHOPS.

[84]  Jean-Luc Dugelay,et al.  People re-identification in camera networks based on probabilistic color histograms , 2011, Electronic Imaging.

[85]  Fabio Roli,et al.  Exploiting Dissimilarity Representations for Person Re-identification , 2011, SIMBAD.

[86]  Fabio Roli,et al.  Fast person re-identification based on dissimilarity representations , 2012, Pattern Recognit. Lett..

[87]  Alessio Del Bue,et al.  Re-identification with RGB-D Sensors , 2012, ECCV Workshops.

[88]  Pietro Perona,et al.  Pedestrian Detection: An Evaluation of the State of the Art , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[89]  Kual-Zheng Lee,et al.  A Simple Calibration Approach to Single View Height Estimation , 2012, 2012 Ninth Conference on Computer and Robot Vision.

[90]  Mohamed Abid,et al.  A fast multi-scale covariance descriptor for object re-identification , 2012, Pattern Recognit. Lett..

[91]  M. Grgic,et al.  Novel pattern recognition-based methods for re-identification in biometric context , 2012, Pattern Recognit. Lett..

[92]  Horst Bischof,et al.  Person Re-identification by Efficient Impostor-Based Metric Learning , 2012, 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance.

[93]  Chunxiao Liu,et al.  Person Re-identification: What Features Are Important? , 2012, ECCV Workshops.

[94]  Niki Martinel,et al.  Re-identify people in wide area camera network , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[95]  Marcel Worring,et al.  Re-identification of persons in multi-camera surveillance under varying viewpoints and illumination , 2012, Defense + Commercial Sensing.

[96]  Alessandro Perina,et al.  Multiple-shot person re-identification by chromatic and epitomic analyses , 2012, Pattern Recognit. Lett..

[97]  Shihong Lao,et al.  Evaluation of color spaces for person re-identification , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[98]  Dong Xu,et al.  Human Gait Recognition Using Patch Distribution Feature and Locality-Constrained Group Sparse Representation , 2012, IEEE Transactions on Image Processing.

[99]  Bingpeng Ma,et al.  Local Descriptors Encoded by Fisher Vectors for Person Re-identification , 2012, ECCV Workshops.

[100]  Fabio Roli,et al.  A General Method for Appearance-Based People Search Based on Textual Queries , 2012, ECCV Workshops.

[101]  Fabio Roli,et al.  Appearance-based people recognition by local dissimilarity representations , 2012, MM&Sec '12.

[102]  Andrew W. Fitzgibbon,et al.  The Vitruvian manifold: Inferring dense correspondences for one-shot human pose estimation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[103]  Michael Lindenbaum,et al.  Learning Implicit Transfer for Person Re-identification , 2012, ECCV Workshops.

[104]  Frédéric Jurie,et al.  PCCA: A new approach for distance learning from sparse pairwise constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[105]  Shishir K. Shah,et al.  Part-based spatio-temporal model for multi-person re-identification , 2012, Pattern Recognit. Lett..

[106]  Gian Luca Foresti,et al.  Multi-signature based person re-identification , 2012 .

[107]  Xiaogang Wang,et al.  Unsupervised Salience Learning for Person Re-identification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[108]  Fabio Roli,et al.  Real-time Appearance-based Person Re-identification Over Multiple KinectTM Cameras , 2013, VISAPP.

[109]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[110]  Riccardo Satta,et al.  Dissimilarity-based people re-identification and search for intelligent video surveillance , 2013 .

[111]  Michael J. Black,et al.  Predicting 3 D People from 2 D Pictures , .