Pedestrian Detection and Tracking from Low-Resolution Unmanned Aerial Vehicle Thermal Imagery

Driven by the prominent thermal signature of humans and following the growing availability of unmanned aerial vehicles (UAVs), more and more research efforts have been focusing on the detection and tracking of pedestrians using thermal infrared images recorded from UAVs. However, pedestrian detection and tracking from the thermal images obtained from UAVs pose many challenges due to the low-resolution of imagery, platform motion, image instability and the relatively small size of the objects. This research tackles these challenges by proposing a pedestrian detection and tracking system. A two-stage blob-based approach is first developed for pedestrian detection. This approach first extracts pedestrian blobs using the regional gradient feature and geometric constraints filtering and then classifies the detected blobs by using a linear Support Vector Machine (SVM) with a hybrid descriptor, which sophisticatedly combines Histogram of Oriented Gradient (HOG) and Discrete Cosine Transform (DCT) features in order to achieve accurate detection. This research further proposes an approach for pedestrian tracking. This approach employs the feature tracker with the update of detected pedestrian location to track pedestrian objects from the registered videos and extracts the motion trajectory data. The proposed detection and tracking approaches have been evaluated by multiple different datasets, and the results illustrate the effectiveness of the proposed methods. This research is expected to significantly benefit many transportation applications, such as the multimodal traffic performance measure, pedestrian behavior study and pedestrian-vehicle crash analysis. Future work will focus on using fused thermal and visual images to further improve the detection efficiency and effectiveness.

[1]  Roland Siegwart,et al.  People detection and tracking from aerial thermal views , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Kang Ryoung Park,et al.  Human Detection Based on the Generation of a Background Image by Using a Far-Infrared Light Camera , 2015, Sensors.

[3]  Riad I. Hammoud,et al.  Pedestrian tracking by fusion of thermal-visible surveillance videos , 2010, Machine Vision and Applications.

[4]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[5]  Pietro Perona,et al.  Pedestrian Detection: An Evaluation of the State of the Art , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[7]  Rainer Stiefelhagen,et al.  Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics , 2008, EURASIP J. Image Video Process..

[8]  Antonio Fernández-Caballero,et al.  Thermal-Infrared Pedestrian ROI Extraction through Thermal and Motion Information Fusion , 2014, Sensors.

[9]  Hui Cheng,et al.  Vehicle and Person Tracking in Aerial Videos , 2007, CLEAR.

[10]  Dariu Gavrila,et al.  Monocular Pedestrian Detection: Survey and Experiments , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Xiaolei Ma,et al.  Mining smart card data for transit riders’ travel patterns , 2013 .

[12]  Norbert Brändle,et al.  Towards complex visual surveillance algorithms on smart cameras , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[13]  Yupin Luo,et al.  Real-Time Pedestrian Detection and Tracking at Nighttime for Driver-Assistance Systems , 2009, IEEE Transactions on Intelligent Transportation Systems.

[14]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[16]  David Gerónimo Gómez,et al.  Survey of Pedestrian Detection for Advanced Driver Assistance Systems , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  M. Shah,et al.  Moving Object Detection and Tracking in Forward Looking Infra-Red Aerial Imagery , 2011 .

[18]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[19]  Xin Li,et al.  Pedestrian detection and tracking in infrared imagery using shape and appearance , 2007, Comput. Vis. Image Underst..

[20]  Mubarak Shah,et al.  Human identity recognition in aerial images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Dragoljub Pokrajac,et al.  People detection in low resolution infrared videos , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[22]  Martin D. Levine,et al.  Face Recognition Using the Discrete Cosine Transform , 2001, International Journal of Computer Vision.

[23]  James W. Davis,et al.  A Two-Stage Template Approach to Person Detection in Thermal Imagery , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[24]  Kang Ryoung Park,et al.  Robust Pedestrian Detection by Combining Visible and Thermal Infrared Cameras , 2015, Sensors.

[25]  Ian D. Reid,et al.  Stable multi-target tracking in real-time surveillance video , 2011, CVPR 2011.

[26]  M. Girish Chandra,et al.  Face recognition using discrete cosine transform and fisher linear discriminant , 2010, 2010 11th International Conference on Control Automation Robotics & Vision.

[27]  Robert A. Schowengerdt,et al.  Airborne video registration and traffic-flow parameter estimation , 2005, IEEE Transactions on Intelligent Transportation Systems.

[28]  Dong Liang,et al.  Robust pedestrian detection in thermal infrared imagery using a shape distribution histogram feature and modified sparse representation classification , 2015, Pattern Recognit..

[29]  Mubarak Shah,et al.  Person Tracking in UAV Video , 2007, CLEAR.

[30]  Xia Liu,et al.  Pedestrian detection and tracking with night vision , 2005, IEEE Transactions on Intelligent Transportation Systems.

[31]  Toby P. Breckon,et al.  Real-time people and vehicle detection from UAV imagery , 2011, Electronic Imaging.

[32]  Jürgen Beyerer,et al.  Low Resolution Person Detection with a Moving Thermal Infrared Camera by Hot Spot Classification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[33]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[34]  Xinkai Wu,et al.  A continuous-flow-intersection-lite design and traffic control for oversaturated bottleneck intersections , 2015 .

[35]  Richard Gran,et al.  On the Convergence of Random Search Algorithms In Continuous Time with Applications to Adaptive Control , 1970, IEEE Trans. Syst. Man Cybern..

[36]  Zheng Liu,et al.  Use of Sparse Representation for Pedestrian Detection in Thermal Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.