Multi-Modal Human Detection from Aerial Views by Fast Shape-Aware Clustering and Classification

Recognizing humans from aerial views represents an increasingly relevant endeavor; a trend mainly driven by the widespread use of unmanned aerial vehicles (UAVs). An accurate and real-time visual human recognition task, however, represents a scientific challenge because typical UAV imaging and computational capabilities and conditions introduce complexities and constraints. Motion blur, the non-specific top-view appearance of humans, low-image resolution and limited onboard computational resources are among the most important limiting factors to be considered. In this paper we propose a run-time-efficient multi-modal detection framework performing clustering and recognition on thermal infrared, passive stereo depth and intensity channels in order to cope with the above complexities and to achieve accurate human detection results. Thermal infrared and depth data are used to generate proposals in combination with an explicit, tree-structured shape representation driven clustering scheme. Generated proposals are used as an input for a discriminatively trained deep classification step to recognize humans. The proposed clustering and classification scheme is validated in qualitative and quantitative terms on four large aerial datasets representing complex (small objects, clutter, occlusions) situations.