Towards Real-Time People Recognition on Aerial Imagery Using Convolutional Neural Networks

People recognition in aerial imagery is a complex problem since it involves cameras angles, which are subject to six degrees of freedom. Traditional classifiers used in pattern matching algorithms are not robust enough to deal with such issues. Thus it is necessary to use other techniques such as deep learning methods, e.g. Convolutional Neural Networks (CNN), in order to cope with such a complex problem. However, in spite of providing potentially better recognition results, deep learning methods demand greater processing time, hindering its use on on-board embedded systems of small Unmanned Aerial Vehicles (UAV). This work is a step towards real-time (and possibly on-board) people recognition on aerial imagery. We propose a new approach based on CNN for real-time people recognition on aerial images obtained from cameras attached to small UAVs. This approach combines images from a low-cost thermal camera to detect candidate objects and CNN for the classification task. We have compared our approach with two traditional pattern matching approaches: saliency map detection and cascade classifiers. The obtained results show that, by using CNN, higher rates of correct classification within 95% of precision are obtained. These results have been obtained within real-time processing of 1.08 fps in the worst case on a hardware without any GPU processing.

[1]  Rogelio Lozano,et al.  Fast and viewpoint robust human detection in uncluttered environments , 2014, 2014 IEEE Visual Communications and Image Processing Conference.

[2]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[3]  Roland Siegwart,et al.  People detection and tracking from aerial thermal views , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[4]  Mark J. Warshawsky,et al.  A Modern Approach , 2005 .

[5]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[6]  Tomaso A. Poggio,et al.  A general framework for object detection , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[7]  Cha Zhang,et al.  Ensemble Machine Learning , 2012 .

[8]  Terrence Fong,et al.  Vehicle detection from aerial imagery , 2011, 2011 IEEE International Conference on Robotics and Automation.

[9]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[10]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[11]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[12]  Matti Pietikäinen,et al.  Performance evaluation of texture measures with classification based on Kullback discrimination of distributions , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[13]  Stephen Cameron,et al.  Multi-modal People Detection from Aerial Video , 2013, CORES.

[14]  Farid Melgani,et al.  Detecting Cars in UAV Images With a Catalog-Based Approach , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Mubarak Shah,et al.  Shadow Casting Out Of Plane (SCOOP) Candidates for Human and Vehicle Detection in Aerial Imagery , 2012, International Journal of Computer Vision.

[16]  K. Upton,et al.  A modern approach , 1995 .

[17]  Heitor Silvério Lopes,et al.  Extracting human attributes using a convolutional neural network approach , 2015, Pattern Recognit. Lett..

[18]  Berin Martini,et al.  Hardware accelerated convolutional neural networks for synthetic vision systems , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[19]  Yann LeCun,et al.  Convolutional networks and applications in vision , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[20]  Piero Toscano,et al.  Intercomparison of UAV, Aircraft and Satellite Remote Sensing Platforms for Precision Viticulture , 2015, Remote. Sens..

[21]  A. Haar.  Zur Theorie der orthogonalen Funktionensysteme , 1910 .

[22]  M. Eichner,et al.  Autonomous Real-time Vehicle Detection from a Medium-Level UAV , 2009 .

[23]  Patrick Doherty,et al.  A UAV Search and Rescue Scenario with Human Body Detection and Geolocalization , 2007, Australian Conference on Artificial Intelligence.

[24]  Anelia Angelova,et al.  Pedestrian detection with a Large-Field-Of-View deep network , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[25]  Bernt Schiele,et al.  Vision based victim detection from unmanned aerial vehicles , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[26]  Toby P. Breckon,et al.  Real-time people and vehicle detection from UAV imagery , 2011, Electronic Imaging.

[27]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[28]  Li WangDong-Chen He,et al.  Texture classification using texture spectrum , 1990, Pattern Recognit..