Robust people detection using depth information from an overhead Time-of-Flight camera

Robust system to detect people only using depth information from a ToF camera.Refined algorithm for determining the regions of interest.Classifier stage to distinguish between people and other objects in the scene.Rigorous experimental procedure shows high performance with a broad user variability.Generated database will be made available to the research community. In this paper we describe a system for the automatic detection of multiple people in a scene, by only using depth information provided by a Time of Flight (ToF) camera placed in overhead position. The main contribution of this work lies in the proposal of a methodology for determining the Regions of Interest (ROI's) and feature extraction, which result in a robust discrimination between people with or without accessories and objects (either static or dynamic), even when people and objects are close together. Since only depth information is used, the developed system guarantees users' privacy. The designed algorithm includes two stages: an online stage, and an offline one. In the offline stage, a new depth image dataset has been recorded and labeled, and the labeled images have been used to train a classifier. The online stage is based on robustly detecting local maximums in the depth image (which are candidates to correspond to the head of the people present in the scene), from which a carefully ROI is defined around each of them. For each ROI, a feature vector is extracted, providing information on the top view of people and objects, including information related to the expected overhead morphology of the head and shoulders. The online stage also includes a pre-filtering process, in order to reduce noise in the depth images. Finally, there is a classification process based on Principal Components Analysis (PCA). The online stage works in real time at an average of 150 fps. In order to evaluate the proposal, a wide experimental validation has been carried out, including different number of people simultaneously present in the scene, as well as people with different heights, complexions, and accessories. The obtained results are very satisfactory, with a 3.1% average error rate.

[1]  John Sell,et al.  The Xbox One System on a Chip and Kinect Sensor , 2014, IEEE Micro.

[2]  Anton Kummert,et al.  People Detection and Tracking from a Top-View Position Using a Time-of-Flight Camera , 2013, MCSS.

[3]  Luigi di Stefano,et al.  People Tracking Using a Time-of-Flight Depth Sensor , 2006, 2006 IEEE International Conference on Video and Signal Based Surveillance.

[4]  J. Goodship,et al.  Centiles for adult head circumference. , 1992, Archives of disease in childhood.

[5]  Álvaro Hernández,et al.  Using PCA in time-of-flight vectors for reflector recognition and 3-D localization , 2005, IEEE Transactions on Robotics.

[6]  Faouzi Alaya Cheikh,et al.  Real-time people counting system using a single video camera , 2008, Electronic Imaging.

[7]  Kin Hong Wong,et al.  Human Tracking and Counting Using the KINECT Range Sensor Based on Adaboost and Kalman Filter , 2013, ISVC.

[8]  Manuel Mazo,et al.  Single frame correction of motion artifacts in PMD-based time of flight cameras , 2014, Image Vis. Comput..

[9]  Tomás Pajdla,et al.  3D with Kinect , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[10]  Sung-Jea Ko,et al.  Robust people counting system based on sensor fusion , 2012, IEEE Transactions on Consumer Electronics.

[11]  Frantisek Galcík,et al.  Real-Time Depth Map Based People Counting , 2013, ACIVS.

[12]  Joaquín Salas,et al.  Counting pedestrians with a zenithal arrangement of depth cameras , 2015, Machine Vision and Applications.

[13]  Andrew Zisserman,et al.  Tracking People by Learning Their Appearance , 2007 .

[14]  Manuel Mazo,et al.  Modeling and correction of multipath interference in time of flight cameras , 2014, Image Vis. Comput..

[15]  Nuno Vasconcelos,et al.  Privacy preserving crowd monitoring: Counting people without people models or tracking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Michael Rauter.  Reliable Human Detection and Tracking in Top-View Depth Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[17]  Mario Vento,et al.  Counting people by RGB or depth overhead cameras , 2016, Pattern Recognit. Lett..

[18]  Tsong-Yi Chen,et al.  A People Counting System Based on Face-Detection , 2010, 2010 Fourth International Conference on Genetic and Evolutionary Computing.

[19]  Ke Zhang,et al.  Counting people in crowded scenes by video analyzing , 2014, 2014 9th IEEE Conference on Industrial Electronics and Applications.

[20]  SuGil Choi,et al.  A method for counting moving and stationary people by interest point classification , 2013, 2013 IEEE International Conference on Image Processing.

[21]  Li Jia,et al.  Using Time-of-Flight Measurements for Privacy-Preserving Tracking in a Smart Room , 2014, IEEE Transactions on Industrial Informatics.

[22]  Alejandro Heredia-Langner,et al.  Standoff human identification using body shape , 2015, 2015 IEEE International Symposium on Technologies for Homeland Security (HST).

[23]  R. Lange,et al.  Solid-state time-of-flight range camera , 2001 .

[24]  Dubravko Culibrk,et al.  K-means based segmentation for real-time zenithal people counting , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[25]  Junjie Yan,et al.  Water Filling: Unsupervised People Counting via Vertical Kinect Sensor , 2012, 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance.

[26]  Anton Kummert,et al.  Applications for a people detection and tracking algorithm using a time-of-flight camera , 2014, Multimedia Tools and Applications.