Object localization in handheld thermal images for fireground understanding

Despite the broad application of the handheld thermal imaging cameras in firefighting, its usage is mostly limited to subjective interpretation by the person carrying the device. As remedies to overcome this limitation, object localization and classification mechanisms could assist the fireground understanding and help with the automated localization, characterization and spatio-temporal (spreading) analysis of the fire. An automated understanding of thermal images can enrich the conventional knowledge-based firefighting techniques by providing the information from the data and sensing-driven approaches. In this work, transfer learning is applied on multi-labeling convolutional neural network architectures for object localization and recognition in monocular visual, infrared and multispectral dynamic images. Furthermore, the possibility of analyzing fire scene images is studied and their current limitations are discussed. Finally, the understanding of the room configuration (i.e., objects location) for indoor localization in reduced visibility environments and the linking with Building Information Models (BIM) are investigated.

[1]  David G. Lilley,et al.  Heat Release Rates of Burning Items in Fires , 2000 .

[2]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[3]  Brian Y. Lattimer,et al.  Real-time probabilistic classification of fire and smoke using thermal imagery for intelligent firefighting robot , 2015 .

[4]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[5]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Vytenis Babrauskas,et al.  Estimating room flashover potential , 1980 .

[7]  Francine K. Amon,et al.  Thermal Imaging Research Needs for First Responders: Workshop Proceedings | NIST , 2005 .

[8]  Michael Bosse,et al.  3D thermal mapping of building interiors using an RGB-D and thermal camera , 2013, 2013 IEEE International Conference on Robotics and Automation.

[9]  Michael Felsberg,et al.  A thermal Object Tracking benchmark , 2015, 2015 12th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[10]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[11]  Steven Verstockt,et al.  On the Use of Real-Time Video to Forecast Fire Growth in Enclosures , 2012 .

[12]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  A. Aydin Alatan,et al.  Object classification in infrared images using deep representations , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[14]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[15]  Gareth Roberts,et al.  Use of Handheld Thermal Imager Data for Airborne Mapping of Fire Radiative Power and Energy and Flame Front Rate of Spread , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Silvio Savarese,et al.  Indoor Scene Understanding with Geometric and Semantic Contexts , 2014, International Journal of Computer Vision.

[17]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Philippe Soussan,et al.  A CMOS-compatible, integrated approach to hyper- and multispectral imaging , 2014, 2014 IEEE International Electron Devices Meeting.

[19]  Håkan Larsson,et al.  Geometric calibration of thermal cameras , 2013, Optics/Photonics in Security and Defence.

[20]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Nicu Sebe,et al.  Where am I in the dark: Exploring active transfer learning on the use of indoor localization based on thermal imaging , 2016, Neurocomputing.

[22]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[23]  Ki-Joune Li,et al.  Integrating IndoorGML and CityGML for Indoor Space , 2014, W2GIS.

[24]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[25]  Margrit Betke,et al.  A Thermal Infrared Video Benchmark for Visual Analysis , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.