Detection and Tracking of Floating Objects Using a UAV with Thermal Camera

This paper develops a vision-based tracking system in unmanned aerial vehicles based on thermal images . The tracking system are tailored toward objects at sea and consists of three main modules that are independent. The first module is an object detection algorithm that uses image analysis techniques to detect marine vessels in thermal images and extract the center of each object. Moreover, as long as the size of the vessel is known or computed in an image where the whole vessel is visible, the center can be identified in situations where only a part of the object is visible. The pixel position of the center is used in a nonlinear state estimator to estimate the position and velocity in a world-fixed coordinate frame. This is called the filtering part of the tracking system. The state estimator is nonlinear because only two coordinates in the world-frame can be computed with the pixel coordinates. This originates from the fact that cameras are bearing-only sensors that are unable to measure range. The last module in the tracking system is data association , which is used to relate new measurements with existing tracks. The tracking system is evaluated in two different case studies. The first case study investigates three different measures for data association in a Monte Carlo simulation. The second case study concerns tracking of a single object in a field experiment, where the object detection algorithm and the filtering part of the tracking system are evaluated. The results show that the modules in the tracking system are reliable with high precision.

[1]  Salah Sukkarieh,et al.  Building a Robust Implementation of Bearing‐only Inertial SLAM for a UAV , 2007, J. Field Robotics.

[2]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[3]  E. Gagnon,et al.  Extended Kalman Filter for State Estimation and Trajectory Prediction of a Moving Object Detected by an Unmanned Aerial Vehicle , 2007, 2007 American Control Conference.

[4]  Randal W. Beard,et al.  A vision-based collision avoidance technique for micro air vehicles using local-level frame mapping and path planning , 2013, Autonomous Robots.

[5]  Thia Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation: Theory, Algorithms and Software , 2001 .

[6]  Thor I. Fossen,et al.  Experimental validation of a uniformly semi-globally exponentially stable non-linear observer for GNSS- and camera-aided inertial navigation for fixed-wing UAVs , 2015, 2015 International Conference on Unmanned Aircraft Systems (ICUAS).

[7]  Tzvetan Semerdjiev,et al.  A study of a target tracking algorithm using global nearest neighbor approach , 2003, CompSysTech '03.

[8]  Timothy W. McLain,et al.  Vision-based Target Geo-location using a Fixed-wing Miniature Air Vehicle , 2006, J. Intell. Robotic Syst..

[9]  Jeremiah Neubert,et al.  On-Board Visual Tracking with Unmanned Aircraft System (UAS) , 2011, ArXiv.

[10]  Tor Arne Johansen,et al.  Automatic detection, classification and tracking of objects in the ocean surface from UAVs using a thermal camera , 2015, 2015 IEEE Aerospace Conference.

[11]  Edmund Førland Brekke,et al.  Compensation of navigation uncertainty for target tracking on a moving platform , 2016, 2016 19th International Conference on Information Fusion (FUSION).

[12]  Emilio Garcia-Fidalgo,et al.  Vessel Inspection: A Micro-Aerial Vehicle-based Approach , 2014, J. Intell. Robotic Syst..

[13]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[14]  Supun Samarasekera,et al.  Aerial video surveillance and exploitation , 2001, Proc. IEEE.

[15]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[16]  Youmin Zhang,et al.  Sense and avoid technologies with applications to unmanned aircraft systems: Review and prospects , 2015 .

[17]  Miguel A. Olivares-Méndez,et al.  A Hierarchical Tracking Strategy for Vision-Based Applications On-Board UAVs , 2013, J. Intell. Robotic Syst..

[18]  Salah Sukkarieh,et al.  Bearing-Only SLAM for an Airborne Vehicle , 2005 .

[19]  X. Rong Li,et al.  Survey of maneuvering target tracking: dynamic models , 2000, SPIE Defense + Commercial Sensing.

[20]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[21]  Peter C. Niedfeldt,et al.  Recursive RANSAC: Multiple Signal Estimation with Outliers , 2013 .

[22]  Salah Sukkarieh,et al.  Airborne simultaneous localisation and map building , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[23]  Tor Arne Johansen,et al.  A ligth-weight thermal camera payload with georeferencing capabilities for small fixed-wing UAVs , 2015, 2015 International Conference on Unmanned Aircraft Systems (ICUAS).

[24]  Elder M. Hemerly,et al.  Automatic Georeferencing of Images Acquired by UAV’s , 2014, Int. J. Autom. Comput..

[25]  Honghai Liu,et al.  Ship Detection and Segmentation Using Image Correlation , 2013, 2013 IEEE International Conference on Systems, Man, and Cybernetics.

[26]  Roger Skjetne,et al.  Ice Observer System For Ice Management Operations , 2011 .

[27]  Tor Arne Johansen,et al.  Tracking of marine surface objects from unmanned aerial vehicles with a pan/tilt unit using a thermal camera and optical flow , 2016, 2016 International Conference on Unmanned Aircraft Systems (ICUAS).

[28]  Ling Shao,et al.  Recent advances and trends in visual tracking: A review , 2011, Neurocomputing.

[29]  Keiichi Abe,et al.  Topological structural analysis of digitized binary images by border following , 1985, Comput. Vis. Graph. Image Process..

[30]  Randal W. Beard,et al.  Multiple target tracking using recursive RANSAC , 2014, 2014 American Control Conference.

[31]  Éric Marchand,et al.  Chasing a moving target from a flying UAV , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[32]  T. Johansen,et al.  Vision-Aided Nonlinear Observer for Fixed-Wing Unmanned Aerial Vehicle Navigation , 2016 .

[33]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[34]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Tristan Perez,et al.  Unmanned aerial surveillance system for hazard collision avoidance in autonomous shipping , 2016, 2016 International Conference on Unmanned Aircraft Systems (ICUAS).