Person Retrieval in Surveillance Videos Using Deep Soft Biometrics

In the world of security and surveillance, it is very common to localize the person in the video. The localization turns to be non-trivial when the search is based on a linguistic query. This is due to the semantic gap between language based query and its processing by a machine. Typically query uses attributes like height, cloth color, cloth type, gender, and hair color, e.g., a tall male with a black t-shirt and pink short. Such attributes are known as soft biometrics. Conventionally, searching the person in the surveillance video is done manually by scouring through hours of videos, which is inefficient and time-consuming. Thus, an automatic person retrieval algorithm is an active area of research.

[1]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[2]  Arun Ross,et al.  What Else Does Your Biometric Data Reveal? A Survey on Soft Biometrics , 2016, IEEE Transactions on Information Forensics and Security.

[3]  Anil K. Jain,et al.  Soft Biometric Traits for Personal Recognition Systems , 2004, ICBA.

[4]  Clinton Fookes,et al.  Semantic Person Retrieval in Surveillance Using Soft Biometrics: AVSS 2018 Challenge II , 2018, 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[5]  Sanjay Chaudhary,et al.  Description Based Person Identification: Use of Clothes Color and Type , 2017, NCVPRIPG.

[6]  Jian Li,et al.  Hierarchical pedestrian attribute recognition based on adaptive region localization , 2017, 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[7]  Mark S. Nixon,et al.  Imputing human descriptions in semantic biometrics , 2010, MiFor '10.

[8]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[9]  Hai Tao,et al.  Evaluating Appearance Models for Recognition, Reacquisition, and Tracking , 2007 .

[10]  Shengcai Liao,et al.  Multi-label CNN based pedestrian attribute learning for soft biometrics , 2015, 2015 International Conference on Biometrics (ICB).

[11]  Shengcai Liao,et al.  Pedestrian Attribute Classification in Surveillance: Database and Evaluation , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[12]  Chunxiao Liu,et al.  Person Re-identification: What Features Are Important? , 2012, ECCV Workshops.

[13]  Rogério Schmidt Feris,et al.  Attribute-based people search in surveillance environments , 2009, 2009 Workshop on Applications of Computer Vision (WACV).

[14]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[15]  Jean-Luc Dugelay,et al.  Bag of soft biometrics for person identification , 2010, Multimedia Tools and Applications.

[16]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Arun Ross,et al.  Soft biometrics for surveillance: an overview , 2013 .

[18]  M.S. Nixon,et al.  The Use of Semantic Human Description as a Soft Biometric , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[19]  Mehul S. Raval,et al.  Person Retrieval in Surveillance Video using Height, Color and Gender , 2018, 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[20]  Shaogang Gong,et al.  Person Re-identification by Attributes , 2012, BMVC.

[21]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[22]  Kaiming He,et al.  Mask R-CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[23]  Yan Wang,et al.  Deep View-Sensitive Pedestrian Attribute Inference in an end-to-end Model , 2017, BMVC.

[24]  Bastian Leibe,et al.  Person Attribute Recognition with a Jointly-Trained Holistic CNN Model , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[25]  Sridha Sridharan,et al.  Searching for people using semantic soft biometric descriptions , 2015, Pattern Recognit. Lett..

[26]  Xiaoou Tang,et al.  Pedestrian Attribute Recognition At Far Distance , 2014, ACM Multimedia.

[27]  Kaiqi Huang,et al.  Multi-attribute learning for pedestrian attribute recognition in surveillance scenarios , 2015, 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR).

[28]  Kaiqi Huang,et al.  A Richly Annotated Pedestrian Dataset for Person Retrieval in Real Surveillance Scenarios , 2019, IEEE Transactions on Image Processing.

[29]  Sridha Sridharan,et al.  Locating People in Video from Semantic Descriptions: A New Database and Approach , 2014, 2014 22nd International Conference on Pattern Recognition.

[30]  Shaogang Gong,et al.  Attribute Recognition by Joint Recurrent Learning of Context and Correlation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[31]  J. Shepherd,et al.  Adult Eyewitness Testimony: Whole body information: Its relevance to eyewitnesses , 1994 .