Multi-sensor fusion for unmanned aerial vehicles based on the combination of filtering and optimization

Accurate and real-time state estimation is the first step to realize safe flight and operation of unmanned aerial vehicles (UAVs). Multi-sensor fusion, e.g., vision, IMU, and GPS, can improve the accuracy of state estimation and even make it work when some sensor is unavailable. Thus, this paper proposes a multi-sensor fusion method based on the combination of filtering and optimization to achieve locally accurate and globally drift-free state estimation. The proposed method has two components, i.e., the Kalman filter and global optimization. The Kalman filter is considered the main structure of the fusion framework, which fuses a local sensor (IMU) and global sensors (aligned global visual inertial odometry, GPS, magnetometer, and barometer sensors) to obtain global state estimation in real time. Global optimization estimates the transformation between local base frame of the visual inertial odometry and global base frame to obtain an accurate global visual estimation. However, given discontinuity of optimization and odometry delay, the aligned visual odometry is then input into the Kalman filter to achieve accurate and drift-free state estimation in real time. Finally, flight and localization tests on a practical UAV were conducted. The experimental results demonstrate the effectiveness and robustness of the proposed multi-sensor fusion method.