Person Re-Identification with RGB-D Camera in Top-View Configuration through Multiple Nearest Neighbor Classifiers and Neighborhood Component Features Selection

Person re-identification is an important topic in retail, scene monitoring, human-computer interaction, people counting, ambient assisted living and many other application fields. A dataset for person re-identification TVPR (Top View Person Re-Identification) based on a number of significant features derived from both depth and color images has been previously built. This dataset uses an RGB-D camera in a top-view configuration to extract anthropometric features for the recognition of people in view of the camera, reducing the problem of occlusions while being privacy preserving. In this paper, we introduce a machine learning method for person re-identification using the TVPR dataset. In particular, we propose the combination of multiple k-nearest neighbor classifiers based on different distance functions and feature subsets derived from depth and color images. Moreover, the neighborhood component feature selection is used to learn the depth features’ weighting vector by minimizing the leave-one-out regularized training error. The classification process is performed by selecting the first passage under the camera for training and using the others as the testing set. Experimental results show that the proposed methodology outperforms standard supervised classifiers widely used for the re-identification task. This improvement encourages the application of this approach in the retail context in order to improve retail analytics, customer service and shopping space management.

[1]  Qi Tian,et al.  Query-adaptive late fusion for image search and person re-identification , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Michele Gorgoglione,et al.  Using context for online customer re-identification , 2016, Expert Syst. Appl..

[3]  Emanuele Frontoni,et al.  Shopper Analytics: A Customer Activity Recognition System Using a Distributed RGB-D Camera Network , 2014, VAAM@ICPR.

[4]  Fabien Moutarde,et al.  Person re-identification in multi-camera system by signature based on interest point descriptors collected on short video sequences , 2008, 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras.

[5]  Simone Calderara,et al.  HECOL: Homography and epipolar-based consistent labeling for outdoor park surveillance , 2008, Comput. Vis. Image Underst..

[6]  László Havasi,et al.  Eigenwalks: walk detection and biometrics from symmetry patterns , 2005, IEEE International Conference on Image Processing 2005.

[7]  Eric Granger,et al.  Progressive Boosting for Class Imbalance , 2017, Expert Syst. Appl..

[8]  Saul Greenberg,et al.  Cross-device interaction via micro-mobility and f-formations , 2012, UIST.

[9]  Ramesh Jain,et al.  Storage and Retrieval for Image and Video Databases III , 1995 .

[10]  Fakhreddine Ababsa,et al.  3D Human Tracking from Depth Cue in a Buying Behavior Analysis Context , 2013, CAIP.

[11]  Emanuele Frontoni,et al.  Mobile robot for retail surveying and inventory using visual and textual analysis of monocular pictures based on deep learning , 2017, 2017 European Conference on Mobile Robots (ECMR).

[12]  Rainer Stiefelhagen,et al.  Interactive person re-identification in TV series , 2010, 2010 International Workshop on Content Based Multimedia Indexing (CBMI).

[13]  Larry S. Davis,et al.  Learning Discriminative Appearance-Based Models Using Partial Least Squares , 2009, 2009 XXII Brazilian Symposium on Computer Graphics and Image Processing.

[14]  Shaogang Gong,et al.  Person Re-Identification by Support Vector Ranking , 2010, BMVC.

[15]  Hai Tao,et al.  Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features , 2008, ECCV.

[16]  Rita Cucchiara,et al.  People reidentification in surveillance and forensics , 2013, ACM Comput. Surv..

[17]  Shaogang Gong,et al.  Reidentification by Relative Distance Comparison , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Wei Yang,et al.  Neighborhood Component Feature Selection for High-Dimensional Data , 2012, J. Comput..

[19]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[20]  Roberto Pierdicca,et al.  Robust and affordable retail customer profiling by vision and radio beacon sensor fusion , 2016, Pattern Recognit. Lett..

[21]  Alessio Del Bue,et al.  Re-identification with RGB-D Sensors , 2012, ECCV Workshops.

[22]  Jun Wang,et al.  Solving the Multiple-Instance Problem: A Lazy Learning Approach , 2000, ICML.

[23]  Roberto Pierdicca,et al.  Low cost embedded system for increasing retail environment intelligence , 2015, 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[24]  Emanuele Frontoni,et al.  Smart Vision System for Shelf Analysis in Intelligent Retail Environments , 2013 .

[25]  Emanuele Frontoni,et al.  Person Re-identification Dataset with RGB-D Camera in a Top-View Configuration , 2016, VAAM/FFER@ICPR.

[26]  Emanuele Frontoni,et al.  Information Management for Intelligent Retail Environment: The Shelf Detector System , 2014, Inf..

[27]  Herman Akdag,et al.  Efficient local monitoring approach for the task of background subtraction , 2017, Eng. Appl. Artif. Intell..

[28]  Stefano Messelodi,et al.  Boosting Fisher vector based scoring functions for person re-identification , 2015, Image Vis. Comput..

[29]  Shihong Lao,et al.  Evaluation of color spaces for person re-identification , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[30]  S. Sathiya Keerthi,et al.  Efficient algorithms for ranking with SVMs , 2010, Information Retrieval.

[31]  Alessandro Perina,et al.  Person re-identification by symmetry-driven accumulation of local features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[32]  Michael Lindenbaum,et al.  Learning Implicit Transfer for Person Re-identification , 2012, ECCV Workshops.

[33]  Bogdan Kwolek,et al.  Fall detection using ceiling-mounted 3D depth camera , 2015, 2014 International Conference on Computer Vision Theory and Applications (VISAPP).

[34]  Emanuele Frontoni,et al.  Pervasive System for Consumer Behaviour Analysis in Retail Environments , 2016, VAAM/FFER@ICPR.

[35]  Emanuele Frontoni,et al.  Modelling and Forecasting Customer Navigation in Intelligent Retail Environments , 2018, J. Intell. Robotic Syst..

[36]  Tiago M. Fragoso,et al.  Bayesian Model Averaging: A Systematic Review and Conceptual Classification , 2015, 1509.08864.

[37]  Hichem Snoussi,et al.  Discriminant quaternion local binary pattern embedding for person re-identification through prototype formation and color categorization , 2017, Eng. Appl. Artif. Intell..

[38]  Jakub Nalepa,et al.  Real-Time People Counting from Depth Images , 2015, BDAS.

[39]  Hamid Beigy,et al.  Detection of evolving concepts in non-stationary data streams: A multiple kernel learning approach , 2018, Expert Syst. Appl..

[40]  Richard I. Hartley,et al.  Person Reidentification Using Spatiotemporal Appearance , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[41]  Rita Cucchiara,et al.  Learning articulated body models for people re-identification , 2013, MM '13.

[42]  Junjie Yan,et al.  Water Filling: Unsupervised People Counting via Vertical Kinect Sensor , 2012, 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance.

[43]  Naohiro Ishii,et al.  Combining Multiple k-Nearest Neighbor Classifiers Using Different Distance Functions , 2004, IDEAL.

[44]  Nikom Suvonvorn,et al.  Top-view Based People Counting Using Mixture of Depth and Color Information , 2013 .

[45]  Horst Bischof,et al.  Large scale metric learning from equivalence constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  David Declercq,et al.  3D facial expression recognition using kernel methods on Riemannian manifold , 2017, Eng. Appl. Artif. Intell..

[47]  Markus A. Stricker,et al.  Similarity of color images , 1995, Electronic Imaging.

[48]  Stephen D. Bay Nearest neighbor classification from multiple feature subsets , 1999, Intell. Data Anal..

[49]  Yang Li,et al.  Multi-shot Re-identification with Random-Projection-Based Random Forests , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[50]  Emanuele Frontoni,et al.  Convolutional Networks for Semantic Heads Segmentation using Top-View Depth Data in Crowded Environment , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[51]  Emanuele Frontoni,et al.  Human activity analysis for in-home fall risk assessment , 2015, 2015 IEEE International Conference on Communication Workshop (ICCW).

[52]  M. D. Ingole,et al.  Content based image retrieval using hybrid features and various distance metric , 2018, Journal of Electrical Systems and Information Technology.

[53]  Vittorio Murino,et al.  Custom Pictorial Structures for Re-identification , 2011, BMVC.

[54]  Fabio Roli,et al.  Fast person re-identification based on dissimilarity representations , 2012, Pattern Recognit. Lett..

[55]  Adrian E. Raftery,et al.  Bayesian Model Averaging: A Tutorial , 2016 .

[56]  Pierre Vandergheynst,et al.  Cascade of descriptors to detect and track objects across any network of cameras , 2010, Comput. Vis. Image Underst..

[57]  Luc Van Gool,et al.  Depth and Appearance for Mobile Scene Analysis , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[58]  Jian-Huang Lai,et al.  Robust Depth-Based Person Re-Identification , 2017, IEEE Transactions on Image Processing.

[59]  Narendra Ahuja,et al.  Pedestrian Recognition with a Learned Metric , 2010, ACCV.

[60]  Alessandro Perina,et al.  Multiple-Shot Person Re-identification by HPE Signature , 2010, 2010 20th International Conference on Pattern Recognition.

[61]  Mubarak Shah,et al.  Modeling inter-camera space-time and appearance relationships for tracking across non-overlapping views , 2008, Comput. Vis. Image Underst..

[62]  Pedro M. Domingos Bayesian Averaging of Classifiers and the Overfitting Problem , 2000, ICML.

[63]  Tarak Gandhi,et al.  Panoramic Appearance Map (PAM) for Multi-camera Based Person Re-identification , 2006, 2006 IEEE International Conference on Video and Signal Based Surveillance.

[64]  Alberto Del Bimbo,et al.  Person Re-Identification by Iterative Re-Weighted Sparse Ranking , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[65]  Shaogang Gong,et al.  Person Re-identification by Video Ranking , 2014, ECCV.

[66]  Rita Cucchiara,et al.  3D Body Model Construction and Matching for Real Time People Re-Identification , 2010, Eurographics Italian Chapter Conference.

[67]  Emanuele Frontoni,et al.  People Detection and Tracking from an RGB-D Camera in Top-View Configuration: Review of Challenges and Applications , 2017, ICIAP Workshops.

[68]  Shishir K. Shah,et al.  A survey of approaches and trends in person re-identification , 2014, Image Vis. Comput..

[69]  Shaogang Gong,et al.  Person re-identification by probabilistic relative distance comparison , 2011, CVPR 2011.

[70]  Zheng Wang,et al.  Zero-Shot Person Re-identification via Cross-View Consistency , 2016, IEEE Transactions on Multimedia.

[71]  Alessandro Perina,et al.  Multiple-shot person re-identification by chromatic and epitomic analyses , 2012, Pattern Recognit. Lett..

[72]  Hai Tao,et al.  Evaluating Appearance Models for Recognition, Reacquisition, and Tracking , 2007 .

[73]  Horst Bischof,et al.  Relaxed Pairwise Learned Metric for Person Re-identification , 2012, ECCV.

[74]  Wasserman,et al.  Bayesian Model Selection and Model Averaging. , 2000, Journal of mathematical psychology.