Graph clustering for weapon discharge event detection and tracking in infrared imagery using deep features

This paper addresses the problem of detecting and tracking weapon discharge event in an Infrared Imagery collection. While most of the prior work in related domains exploits the vast amount of complementary in- formation available from both visible-band (EO) and Infrared (IR) image (or video sequences), we handle the problem of recognizing human pose and activity detection exclusively in thermal (IR) images or videos. The task is primarily two-fold: 1) locating the individual in the scene from IR imagery, and 2) identifying the correct pose of the human individual (i.e. presence or absence of weapon discharge activity or intent). An efficient graph-based shortlisting strategy for identifying candidate regions of interest in the IR image utilizes both image saliency and mutual similarities from the initial list of the top scored proposals of a given query frame, which ensures an improved performance for both detection and recognition simultaneously and reduced false alarms. The proposed search strategy offers an efficient feature extraction scheme that can capture the maximum amount of object structural information by defining a region- based deep shape descriptor representing each object of interest present in the scene. Therefore, our solution is capable of handling the fundamental incompleteness of the IR imageries for which the conventional deep features optimized on the natural color images in Imagenet are not quite suitable. Our preliminary experiments on the OSU weapon dataset demonstrates significant success in automated recognition of weapon discharge events from IR imagery.