Detection of Inflatable Boats and People in Thermal Infrared with Deep Learning Methods

Smuggling of drugs and cigarettes in small inflatable boats across border rivers is a serious threat to the EU’s financial interests. Early detection of such threats is challenging due to difficult and changing environmental conditions. This study reports on the automatic detection of small inflatable boats and people in a rough wild terrain in the infrared thermal domain. Three acquisition campaigns were carried out during spring, summer, and fall under various weather conditions. Three deep learning algorithms, namely, YOLOv2, YOLOv3, and Faster R-CNN working with six different feature extraction neural networks were trained and evaluated in terms of performance and processing time. The best performance was achieved with Faster R-CNN with ResNet101, however, processing requires a long time and a powerful graphics processing unit.

[1]  Hua Yu,et al.  A direct LDA algorithm for high-dimensional data - with application to face recognition , 2001, Pattern Recognit..

[2]  Pengwen Xiong,et al.  Robust Visual Ship Tracking with an Ensemble Framework via Multi-View Learning and Wavelet Filter , 2020, Sensors.

[3]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Ivan Laptev,et al.  Context-Aware CNNs for Person Head Detection , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[5]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[6]  Amir Averbuch,et al.  Acoustic detection and classification of river boats , 2011 .

[7]  W. Pedrycz,et al.  Exploring thermal images for object detection in underexposure regions for autonomous driving , 2020, Appl. Soft Comput..

[8]  Matti Pietikäinen,et al.  IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2009, TPAMI-2008-09-0620 1 WLD: A Robust Local Image Descriptor , 2022 .

[9]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[11]  Hong-Yuan Mark Liao,et al.  YOLOv4: Optimal Speed and Accuracy of Object Detection , 2020, ArXiv.

[12]  Sorin Grigorescu,et al.  A Survey of Deep Learning Techniques for Autonomous Driving , 2020, J. Field Robotics.

[13]  Anne-Claire Boury-Brisset,et al.  Ship Classification Using Deep Learning Techniques for Maritime Target Tracking , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[14]  Javier Ruiz-del-Solar,et al.  Face recognition using thermal infrared images for Human-Robot Interaction applications: A comparative study , 2009, 2009 6th Latin American Robotics Symposium (LARS 2009).

[15]  Natalia Wawrzyniak,et al.  Vessel Detection and Tracking Method Based on Video Surveillance , 2019, Sensors.

[16]  Tor Arne Johansen,et al.  Detectability of Objects at the Sea Surface in Visible Light and Thermal Camera Images , 2018, 2018 OCEANS - MTS/IEEE Kobe Techno-Oceans (OTO).

[17]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Euntai Kim,et al.  Probabilistic Ship Detection and Classification Using Deep Learning , 2018, Applied Sciences.

[19]  Yuanyuan Ji,et al.  An infrared maritime target detection algorithm applicable to heavy sea fog , 2015 .

[20]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Andrzej Stateczny,et al.  Remote Sensing in Vessel Detection and Navigation , 2020, Sensors.

[22]  J. Edward Jackson,et al.  A User's Guide to Principal Components: Jackson/User's Guide to Principal Components , 2004 .

[23]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[24]  Mogens Blanke,et al.  Assessing Deep-learning Methods for Object Detection at Sea from LWIR Images , 2019, IFAC-PapersOnLine.

[25]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[26]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[27]  Yangfan Huang,et al.  Thermal Infrared Small Ship Detection in Sea Clutter Based on Morphological Reconstruction and Multi-Feature Analysis , 2019, Applied Sciences.

[28]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[29]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[30]  Silvio Savarese,et al.  Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Xue Yuan,et al.  TIRNet: Object detection in thermal infrared images for autonomous driving , 2020, Applied Intelligence.

[32]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Yeongjae Cheon,et al.  PVANET: Deep but Lightweight Neural Networks for Real-time Object Detection , 2016, ArXiv.

[34]  Nelson F. F. Ebecken,et al.  A SURVEY ON VIDEO DETECTION AND TRACKING OF MARITIME VESSELS , 2014 .

[35]  Jih-Gau Juang,et al.  Drone patrol using thermal imaging for object detection , 2020, Optical Engineering + Applications.

[36]  J. Edward Jackson,et al.  A User's Guide to Principal Components. , 1991 .

[37]  Matti Pietikäinen,et al.  Performance evaluation of texture measures with classification based on Kullback discrimination of distributions , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[38]  Olivier Morère,et al.  Maritime Vessel Images Classification Using Deep Convolutional Neural Networks , 2015, SoICT.

[39]  Qiang Ji,et al.  A Comparative Study of Local Matching Approach for Face Recognition , 2007, IEEE Transactions on Image Processing.

[40]  Miran Pobar,et al.  Thermal Object Detection in Difficult Weather Conditions Using YOLO , 2020, IEEE Access.