Space-time Histograms And Their Application To Person Re-identification In TV Shows

The annotation of video streams by automatic content analysis is a growing field of research. The possibility of recognising persons appearing in TV shows allows to automatically structure ever-growing video archives. We propose a new descriptor to re-identify persons featured in videos, that is to say, to spot all occurrences of persons throughout a video. Our approach is dynamic as it benefits from motion information contained in videos, whereas the static approaches are solely based on still images. We extract person-tracks from videos and match them using a new descriptor and its associated similarity measure: the space-time histogram. The originality of our approach is the integration of temporal data into the descriptor. Experiments show that it provides a better estimation of the similarity between person-tracks. Our contribution has been evaluated using a corpus of real life french TV shows broadcasted on BFMTV and LCP TV channels and on some annotated episodes from "Buffy: the Vampire Slayer". Experimental results show that our approach significantly improves the precision of the re-identification process thanks to the use of the temporal dimension.

[1]  Cordelia Schmid,et al.  Is that you? Metric learning approaches for face identification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[2]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[3]  Fabien Moutarde,et al.  Person re-identification in multi-camera system by signature based on interest point descriptors collected on short video sequences , 2008, 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras.

[4]  Hai-Miao Hu,et al.  A person re-identification algorithm based on color topology , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[5]  Stanley T. Birchfield,et al.  Spatiograms versus histograms for region-based tracking , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[6]  Cordelia Schmid,et al.  Unsupervised metric learning for face identification in TV video , 2011, 2011 International Conference on Computer Vision.

[7]  Eamonn J. Keogh,et al.  Experimental comparison of representation methods and distance measures for time series data , 2010, Data Mining and Knowledge Discovery.

[8]  Duy-Dinh Le,et al.  Face Detection, Tracking, and Recognition for Broadcast Video , 2008, Encyclopedia of Multimedia.

[9]  Horst Bischof,et al.  Person Re-identification by Descriptive and Discriminative Classification , 2011, SCIA.

[10]  Olivier Galibert,et al.  The First Official REPERE Evaluation , 2013, SLAM@INTERSPEECH.

[11]  Larry S. Davis,et al.  A Robust and Scalable Approach to Face Identification , 2010, ECCV.

[12]  Yu-Chiang Frank Wang,et al.  Exploiting low-rank structures from cross-camera images for robust person re-identification , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[13]  Vijay V. Raghavan,et al.  A critical investigation of recall and precision as measures of retrieval system performance , 1989, TOIS.

[14]  Paul A. Viola,et al.  Robust Real-time Object Detection , 2001 .

[15]  Rémi Ronfard,et al.  Detecting and Naming Actors in Movies Using Generative Appearance Models , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Osama Masoud,et al.  Detection of loitering individuals in public transportation areas , 2005, IEEE Transactions on Intelligent Transportation Systems.

[17]  Louahdi Khoudour,et al.  Video Sequences Association for People Re-identification across Multiple Non-overlapping Cameras , 2009, ICIAP.

[18]  Andrew Zisserman,et al.  Hello! My name is... Buffy'' -- Automatic Naming of Characters in TV Video , 2006, BMVC.

[19]  Jean-Marc Odobez,et al.  Fusing matching and biometric similarity measures for face diarization in video , 2013, ICMR '13.

[20]  Andrew Zisserman,et al.  “Who are you?” - Learning person specific classifiers from video , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Duy-Dinh Le,et al.  Robust Face Track Finding in Video Using Tracked Points , 2008, 2008 IEEE International Conference on Signal Image Technology and Internet Based Systems.

[22]  Horst Bischof,et al.  Learning to recognize faces from videos and weakly related information cues , 2011, 2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[23]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[25]  Shishir K. Shah,et al.  A survey of approaches and trends in person re-identification , 2014, Image Vis. Comput..

[26]  Shaogang Gong,et al.  Person re-identification by probabilistic relative distance comparison , 2011, CVPR 2011.

[27]  Hicham G. Elmongui,et al.  Spatio-temporal Histograms , 2005, SSTD.

[28]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[29]  Pierre Tirilly,et al.  Introducing FoxPersonTracks: A benchmark for person re-identification from TV broadcast shows , 2015, 2015 13th International Workshop on Content-Based Multimedia Indexing (CBMI).

[30]  Hans-Peter Kriegel,et al.  Advances in Spatial and Temporal Databases , 2013, Lecture Notes in Computer Science.

[31]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..