HARD-PnP: PnP Optimization Using a Hybrid Approximate Representation

This paper proposes a Hybrid Approximate Representation (HAR) based on unifying several efficient approximations of the generalized reprojection error (which is known as the <italic>gold standard</italic> for multiview geometry). The HAR is an over-parameterization scheme where the approximation is applied simultaneously in multiple parameter spaces. A joint minimization scheme “HAR-Descent” can then solve the PnP problem efficiently, while remaining robust to approximation errors and local minima. The technique is evaluated extensively, including numerous synthetic benchmark protocols and the real-world data evaluations used in previous works. The proposed technique was found to have runtime complexity comparable to the fastest <inline-formula><tex-math notation="LaTeX">$O(n)$</tex-math><alternatives><inline-graphic xlink:href="hadfield-ieq1-2806446.gif"/></alternatives></inline-formula> techniques, and up to 10 times faster than current state of the art minimization approaches. In addition, the accuracy exceeds that of all 9 previous techniques tested, providing definitive state of the art performance on the benchmarks, across all 90 of the experiments in the paper and supplementary material, which can be found on the Computer Society Digital Library at <uri>http://doi.ieeecomputersociety.org/10.1109/TPAMI.2018.2806446</uri>.