Efficient Real-Time Radial Distortion Correction for UAVs

In this paper we present a novel algorithm for onboard radial distortion correction for unmanned aerial vehicles (UAVs) equipped with an inertial measurement unit (IMU), that runs in real-time. This approach makes calibration procedures redundant, thus allowing for exchange of optics extemporaneously. By utilizing the IMU data, the cameras can be aligned with the gravity direction. This allows us to work with fewer degrees of freedom, and opens up for further intrinsic calibration. We propose a fast and robust minimal solver for simultaneously estimating the focal length, radial distortion profile and motion parameters from homographies. The proposed solver is tested on both synthetic and real data, and perform better or on par with state-of-the-art methods relying on pre-calibration procedures. Code available at: https://github.com/marcusvaltonen/HomLib.1

[1]  Zuzana Kukelova,et al.  Making minimal solvers fast , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Jörn Ostermann,et al.  In-loop radial distortion compensation for long-term mosaicing of aerial videos , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[3]  Sebastian Madgwick,et al.  Estimation of IMU and MARG orientation using a gradient descent algorithm , 2011, 2011 IEEE International Conference on Rehabilitation Robotics.

[4]  Matthew A. Brown,et al.  Minimal Solutions for Panoramic Stitching with Radial Distortion , 2009, BMVC.

[5]  Pascal Vasseur,et al.  Visual Odometry Using a Homography Formulation with Decoupled Rotation and Translation Estimation Using Minimal Solutions , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[6]  Zuzana Kukelova,et al.  A Minimal Solution to Radial Distortion Autocalibration , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Zuzana Kukelova,et al.  Beyond Grobner Bases: Basis Selection for Minimal Solvers , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  Dean Brown,et al.  Decentering distortion of lenses , 1966 .

[9]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Zuzana Kukelova,et al.  Radial Distortion Triangulation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Friedrich Fraundorfer,et al.  Minimal solutions for the rotational alignment of IMU-camera systems using homography constraints , 2018, Comput. Vis. Image Underst..

[12]  M. Pollefeys,et al.  Homography based visual odometry with known vertical direction and weak Manhattan world assumption , 2012 .

[13]  Jiri Matas,et al.  Locally Optimized RANSAC , 2003, DAGM-Symposium.

[14]  Pascal Vasseur,et al.  Homography Based Egomotion Estimation with a Common Direction , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  H. M. Möller,et al.  Multivariate polynomial equations with multiple zeros solved by matrix eigenproblems , 1995 .

[16]  Jean Ponce,et al.  An Efficient Solution to the Homography-Based Relative Pose Problem With a Common Reference Direction , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[17]  Klas Josephson,et al.  Pose estimation with radial distortion and unknown focal length , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Viktor Larsson,et al.  Polynomial Solvers for Saturated Ideals , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[19]  Amir Hashemi,et al.  A New Solution to the Relative Orientation Problem Using Only 3 Points and the Vertical Direction , 2009, Journal of Mathematical Imaging and Vision.

[20]  Anders Heyden,et al.  Minimal Solvers for Indoor UAV Positioning , 2020, ArXiv.

[21]  Matthew Turk,et al.  Solving for Relative Pose with a Partially Known Rotation is a Quadratic Eigenvalue Problem , 2014, 2014 2nd International Conference on 3D Vision.

[22]  Zuzana Kukelova,et al.  New Efficient Solution to the Absolute Pose Problem for Camera with Unknown Focal Length and Radial Distortion , 2010, ACCV.

[23]  Marc Pollefeys,et al.  A Minimal Case Solution to the Calibrated Relative Pose Problem for the Case of Two Known Orientation Angles , 2010, ECCV.

[24]  Yubin Kuang,et al.  Minimal Solvers for Relative Pose with a Single Unknown Radial Distortion , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Zuzana Kukelova,et al.  Real-Time Solution to the Absolute Pose Problem with Unknown Radial Distortion and Focal Length , 2013, 2013 IEEE International Conference on Computer Vision.

[26]  Marcus Valtonen Örnhag.  Radially Distorted Planar Motion Compatible Homographies , 2020, ICPRAM.

[27]  Zuzana Kukelova,et al.  Fast and robust numerical solutions to minimal problems for cameras with radial distortion , 2008, CVPR.

[28]  Richard I. Hartley,et al.  In Defense of the Eight-Point Algorithm , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Zuzana Kukelova,et al.  Rectification from Radially-Distorted Scales , 2018, ACCV.

[30]  Stergios I. Roumeliotis,et al.  Two Efficient Solutions for Visual Odometry Using Directional Correspondence , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Zuzana Kukelova,et al.  Radial distortion homography , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Janne Heikkilä,et al.  A Sparse Resultant Based Method for Efficient Minimal Solvers , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Zuzana Kukelova,et al.  Camera Pose Estimation with Unknown Principal Point , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[34]  Martin Byröd,et al.  Fast and Stable Polynomial Equation Solving and Its Application to Computer Vision , 2009, International Journal of Computer Vision.

[35]  Zuzana Kukelova,et al.  A Clever Elimination Strategy for Efficient Minimal Solvers , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Andrew W. Fitzgibbon,et al.  Simultaneous linear estimation of multiple view geometry and lens distortion , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[37]  Zuzana Kukelova,et al.  Making Minimal Solvers for Absolute Pose Estimation Compact and Robust , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[38]  Zuzana Kukelova,et al.  Automatic Generator of Minimal Problem Solvers , 2008, ECCV.

[39]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[40]  Zuzana Kukelova,et al.  Radially-Distorted Conjugate Translations , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  Torsten Sattler,et al.  Revisiting Radial Distortion Absolute Pose , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[42]  Ji Zhao,et al.  Minimal Solutions for Relative Pose With a Single Affine Correspondence , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Janne Heikkilä,et al.  Computing stable resultant-based minimal solvers by hiding a variable , 2020, 2020 25th International Conference on Pattern Recognition (ICPR).

[44]  I HartleyRichard.  In Defense of the Eight-Point Algorithm , 1997 .

[45]  Viktor Larsson,et al.  Uncovering Symmetries in Polynomial Systems , 2016, ECCV.

[46]  Torsten Sattler,et al.  Why Having 10,000 Parameters in Your Camera Model Is Better Than Twelve , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Zuzana Kukelova,et al.  Fast and robust numerical solutions to minimal problems for cameras with radial distortion , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.