Object Classification in Thermal Images using Convolutional Neural Networks for Search and Rescue Missions with Unmanned Aerial Systems

In recent years, the use of Unmanned Aerial Systems (UAS) has become commonplace in a wide variety of tasks due to their relatively low cost and ease of operation. In this paper, we explore the use of UAS in maritime Search And Rescue (SAR) missions by using experimental data to detect and classify objects at the sea surface. The objects are chosen as common objects present in maritime SAR missions: a boat, a pallet, a human, and a buoy. The data consists of thermal images and a Gaussian Mixture Model (GMM) is used to discriminate foreground objects from the background. Then, bounding boxes containing the object are defined and used to train a Convolutional Neural Network (CNN). The CNN achieves the average accuracy of 92.5% when evaluating a testing dataset.

[1]  Tomaso A. Poggio,et al.  A Trainable System for Object Detection , 2000, International Journal of Computer Vision.

[2]  Menglong Yan,et al.  Semantic Segmentation of Aerial Images With Shuffling Convolutional Neural Networks , 2018, IEEE Geoscience and Remote Sensing Letters.

[3]  Rik Van de Walle,et al.  Deep Learning for Infrared Thermal Image Based Machine Health Monitoring , 2017, IEEE/ASME Transactions on Mechatronics.

[4]  Dimitris A. Pados,et al.  On overfitting, generalization, and randomly expanded training sets , 2000, IEEE Trans. Neural Networks Learn. Syst..

[5]  Tor Arne Johansen,et al.  Automatic detection, classification and tracking of objects in the ocean surface from UAVs using a thermal camera , 2015, 2015 IEEE Aerospace Conference.

[6]  Lior Rokach,et al.  Pattern Classification Using Ensemble Methods , 2009, Series in Machine Perception and Artificial Intelligence.

[7]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[8]  Hong Tang,et al.  DropBand: A Simple and Effective Method for Promoting the Scene Classification Accuracy of Convolutional Neural Networks for VHR Remote Sensing Imagery , 2018, IEEE Geoscience and Remote Sensing Letters.

[9]  Marc Van Droogenbroeck,et al.  An Evaluation of Pixel-Based Methods for the Detection of Floating Objects on the Sea Surface , 2010, EURASIP J. Adv. Signal Process..

[10]  Ewert Bengtsson,et al.  Deep Convolutional Neural Networks for Detecting Cellular Changes Due to Malignancy , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[11]  Farid Melgani,et al.  Convolutional neural networks for near real-time object detection from UAV imagery in avalanche search and rescue operations , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[12]  Simon J. D. Prince,et al.  Computer Vision: Models, Learning, and Inference , 2012 .

[13]  Frédéric Maire,et al.  A Convolutional Neural Network for Automatic Analysis of Aerial Imagery , 2014, 2014 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[14]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[15]  Tianming Liu,et al.  Learning to Predict Eye Fixations via Multiresolution Convolutional Neural Networks , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Dong Yu,et al.  Automatic Speech Recognition: A Deep Learning Approach , 2014 .

[17]  Domenico Velotto,et al.  Ship Classification in TerraSAR-X Images With Convolutional Neural Networks , 2018, IEEE Journal of Oceanic Engineering.

[18]  Michele Volpi,et al.  Fast animal detection in UAV images using convolutional neural networks , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[19]  Maryam Rahnemoonfar,et al.  Real-time scene understanding for UAV imagery based on deep convolutional neural networks , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[20]  Lutz Prechelt,et al.  Early Stopping-But When? , 1996, Neural Networks: Tricks of the Trade.

[21]  Zheng Liu,et al.  Pedestrian detection in thermal images using adaptive fuzzy C-means clustering and convolutional neural networks , 2015, 2015 14th IAPR International Conference on Machine Vision Applications (MVA).

[22]  Klaus-Robert Müller,et al.  Asymptotic statistical theory of overtraining and cross-validation , 1997, IEEE Trans. Neural Networks.

[23]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[24]  Takeshi Nakazawa,et al.  Wafer Map Defect Pattern Classification and Image Retrieval Using Convolutional Neural Network , 2018, IEEE Transactions on Semiconductor Manufacturing.

[25]  Shin-Jye Lee,et al.  Image Classification Based on the Boost Convolutional Neural Network , 2018, IEEE Access.

[26]  S. Lim,et al.  Stable improved softmax using constant normalisation , 2017 .

[27]  Alexandre Bernardino,et al.  Aerial Detection in Maritime Scenarios Using Convolutional Neural Networks , 2016, ACIVS.

[28]  Kaisa Liimatainen,et al.  Dual Structured Convolutional Neural Network with Feature Augmentation for Quantitative Characterization of Tissue Histology , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[29]  Zhongfeng Wang,et al.  An Energy-Efficient Architecture for Binary Weight Convolutional Neural Networks , 2018, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.