Mitigating the effects of boom occlusion on automated aerial refueling through shadow volumes

In-flight refueling of unmanned aerial vehicles (UAVs) is critical to the United States Air Force (USAF). However, the large communication latency between a ground-based operator and his/her remote UAV makes docking with a refueling tanker unsafe. This latency may be mitigated by leveraging a tanker-centric stereo vision system. The vision system observes and computes an approaching receiver’s relative position and orientation offering a low-latency, high frequency docking solution. Unfortunately, the boom – an articulated refueling arm responsible for physically pumping fuel into the receiver – occludes large portions of the receiver especially as the receiver approaches and docks with the tanker. The vision system must be able to compensate for the boom’s occlusion of the receiver aircraft. We present a novel algorithm for mitigating the negative effects of boom occlusion in stereo-based aerial environments. Our algorithm dynamically compensates for occluded receiver geometry by transforming the occluded areas into shadow volumes. These shadow volumes are then used to cull hidden geometry that is traditionally consumed, in error, by the vision processing and point registration pipeline. Our algorithm improves computer-vision pose estimates by 44% over a naïve approach without shadow volume culling.