Onboard Detection-Tracking-Localization

This article investigates long-term positioning of moving objects by monocular vision of a miniature fixed-wing unmanned aerial vehicle. It is challenging to perform a real-time onboard vision processing task, due to the strict payload capacity and power budget limitations of microflying vehicles. We propose a parallel onboard architecture that explicitly decouples the long-term positioning task into iteratively operated detection, tracking, and localization. The proposed approach is eventually called onboard detection-tracking-localization, namely oDTL. The detector automatically extracts and identifies the object from image frames captured at in-flight durations. A learning-based network is constructed to improve detection accuracy and robustness against ever-changing outdoor illumination conditions and flying viewpoints. The tracker follows the object within specified region-of-interest from frame to frame with lower computing consumption. To further reduce target-losing rate, a concept of blind zone is proposed and applied, and its boundaries in sequential images are also theoretically inferred. The position estimator maps the flying vehicle pose, the image coordinates, and calibration specifications into real-world positions of the moving target. An extended Kalman filter is developed for rough position estimation, and a smooth module is introduced for the refinement of the position. Three offline comparative experiments and three online experiments have been conducted respectively to testify the real-time capability of our approach. The collected experimental results also demonstrate the feasible accuracy and robustness of the overall solution within the specified flying onboard scenarios.