Efficient Real-Time Radial Distortion Correction for UAVs

In this paper we present a novel algorithm for onboard radial distortion correction for unmanned aerial vehicles (UAVs) equipped with an inertial measurement unit (IMU), that runs in real-time. This approach makes calibration procedures redundant, thus allowing for exchange of optics extemporaneously. By utilizing the IMU data, the cameras can be aligned with the gravity direction. This allows us to work with fewer degrees of freedom, and opens up for further intrinsic calibration. We propose a fast and robust minimal solver for simultaneously estimating the focal length, radial distortion profile and motion parameters from homographies. The proposed solver is tested on both synthetic and real data, and perform better or on par with state-of-the-art methods relying on pre-calibration procedures. Code available at: https://github.com/marcusvaltonen/HomLib.1

[1]  Dean Brown,et al.  Decentering distortion of lenses , 1966 .

[2]  H. M. Möller,et al.  Multivariate polynomial equations with multiple zeros solved by matrix eigenproblems , 1995 .

[3]  Richard I. Hartley,et al.  In Defense of the Eight-Point Algorithm , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[5]  Andrew W. Fitzgibbon,et al.  Simultaneous linear estimation of multiple view geometry and lens distortion , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[6]  Jiri Matas,et al.  Locally Optimized RANSAC , 2003, DAGM-Symposium.

[7]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Zuzana Kukelova,et al.  Fast and robust numerical solutions to minimal problems for cameras with radial distortion , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Zuzana Kukelova,et al.  Automatic Generator of Minimal Problem Solvers , 2008, ECCV.

[10]  Zuzana Kukelova,et al.  Fast and robust numerical solutions to minimal problems for cameras with radial distortion , 2008, CVPR.

[11]  Martin Byröd,et al.  Fast and Stable Polynomial Equation Solving and Its Application to Computer Vision , 2009, International Journal of Computer Vision.

[12]  Klas Josephson,et al.  Pose estimation with radial distortion and unknown focal length , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Matthew A. Brown,et al.  Minimal Solutions for Panoramic Stitching with Radial Distortion , 2009, BMVC.

[14]  Zuzana Kukelova,et al.  New Efficient Solution to the Absolute Pose Problem for Camera with Unknown Focal Length and Radial Distortion , 2010, ACCV.

[15]  Marc Pollefeys,et al.  A Minimal Case Solution to the Calibrated Relative Pose Problem for the Case of Two Known Orientation Angles , 2010, ECCV.

[16]  Amir Hashemi,et al.  A New Solution to the Relative Orientation Problem Using Only 3 Points and the Vertical Direction , 2009, Journal of Mathematical Imaging and Vision.

[17]  Sebastian Madgwick,et al.  Estimation of IMU and MARG orientation using a gradient descent algorithm , 2011, 2011 IEEE International Conference on Rehabilitation Robotics.

[18]  Zuzana Kukelova,et al.  A Minimal Solution to Radial Distortion Autocalibration , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Stergios I. Roumeliotis,et al.  Two Efficient Solutions for Visual Odometry Using Directional Correspondence , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Zuzana Kukelova,et al.  Making minimal solvers fast , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  M. Pollefeys,et al.  Homography based visual odometry with known vertical direction and weak Manhattan world assumption , 2012 .

[22]  Zuzana Kukelova,et al.  Real-Time Solution to the Absolute Pose Problem with Unknown Radial Distortion and Focal Length , 2013, 2013 IEEE International Conference on Computer Vision.

[23]  Matthew Turk,et al.  Solving for Relative Pose with a Partially Known Rotation is a Quadratic Eigenvalue Problem , 2014, 2014 2nd International Conference on 3D Vision.

[24]  Yubin Kuang,et al.  Minimal Solvers for Relative Pose with a Single Unknown Radial Distortion , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Zuzana Kukelova,et al.  Radial distortion homography , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Viktor Larsson,et al.  Uncovering Symmetries in Polynomial Systems , 2016, ECCV.

[27]  Jörn Ostermann,et al.  In-loop radial distortion compensation for long-term mosaicing of aerial videos , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[28]  Viktor Larsson,et al.  Polynomial Solvers for Saturated Ideals , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  Zuzana Kukelova,et al.  Making Minimal Solvers for Absolute Pose Estimation Compact and Robust , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[30]  Zuzana Kukelova,et al.  A Clever Elimination Strategy for Efficient Minimal Solvers , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Pascal Vasseur,et al.  Homography Based Egomotion Estimation with a Common Direction , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Friedrich Fraundorfer,et al.  Minimal solutions for the rotational alignment of IMU-camera systems using homography constraints , 2018, Comput. Vis. Image Underst..

[33]  Zuzana Kukelova,et al.  Radially-Distorted Conjugate Translations , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[34]  Zuzana Kukelova,et al.  Beyond Grobner Bases: Basis Selection for Minimal Solvers , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[35]  Zuzana Kukelova,et al.  Rectification from Radially-Distorted Scales , 2018, ACCV.

[36]  Zuzana Kukelova,et al.  Camera Pose Estimation with Unknown Principal Point , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[37]  Pascal Vasseur,et al.  Visual Odometry Using a Homography Formulation with Decoupled Rotation and Translation Estimation Using Minimal Solutions , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[38]  Zuzana Kukelova,et al.  Radial Distortion Triangulation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Jean Ponce,et al.  An Efficient Solution to the Homography-Based Relative Pose Problem With a Common Reference Direction , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[40]  Torsten Sattler,et al.  Revisiting Radial Distortion Absolute Pose , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[41]  Janne Heikkilä,et al.  A Sparse Resultant Based Method for Efficient Minimal Solvers , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Torsten Sattler,et al.  Why Having 10,000 Parameters in Your Camera Model Is Better Than Twelve , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Marcus Valtonen Örnhag.  Radially Distorted Planar Motion Compatible Homographies , 2020, ICPRAM.

[44]  Minimal Solutions for Relative Pose With a Single Affine Correspondence , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Anders Heyden,et al.  Minimal Solvers for Indoor UAV Positioning , 2020, ArXiv.

[46]  Janne Heikkilä,et al.  Computing stable resultant-based minimal solvers by hiding a variable , 2020, 2020 25th International Conference on Pattern Recognition (ICPR).