Rolling Shutter Camera Absolute Pose

We present minimal, non-iterative solutions to the absolute pose problem for images from rolling shutter cameras. The absolute pose problem is a key problem in computer vision and rolling shutter is present in a vast majority of today's digital cameras. We discuss several camera motion models and propose two feasible rolling shutter camera models for a polynomial solver. In previous work a linearized camera model was used that required an initial estimate of the camera orientation. We show how to simplify the system of equations and make this solver faster. Furthermore, we present a first solution of the non-linearized camera orientation model using the Cayley parameterization. The new solver does not require any initial camera orientation estimate and therefore serves as a standalone solution to the rolling shutter camera pose problem from six 2D-to-3D correspondences. We show that our algorithms outperform P3P followed by a non-linear refinement using a rolling shutter model.

[1]  V. Lepetit,et al.  EPnP: An Accurate O(n) Solution to the PnP Problem , 2009, International Journal of Computer Vision.

[2]  Zuzana Kukelova,et al.  Making minimal solvers fast , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  B. Triggs,et al.  Camera Pose Revisited -- New Linear Algorithms , 2000 .

[4]  Jan-Michael Frahm,et al.  Structure-from-Motion Revisited , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  David W. Murray,et al.  Parallel Tracking and Mapping on a camera phone , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[6]  Torsten Sattler,et al.  Efficient & Effective Prioritized Matching for Large-Scale Image-Based Localization , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Ludovic Magerand,et al.  Global Optimization of Object Pose and Motion from a Single Rolling Shutter Image with Automatic 2D-3D Matching , 2012, ECCV.

[8]  James R. Bergen,et al.  Visual odometry , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[9]  Gaku Nakano,et al.  Globally Optimal DLS Method for PnP Problem with Cayley parameterization , 2015, BMVC.

[10]  Didier Henrion,et al.  GloptiPoly 3: moments, optimization and semidefinite programming , 2007, Optim. Methods Softw..

[11]  Ken Shoemake,et al.  Animating rotation with quaternion curves , 1985, SIGGRAPH.

[12]  Marina Weber,et al.  Using Algebraic Geometry , 2016 .

[13]  Friedrich Fraundorfer,et al.  Visual Odometry Part I: The First 30 Years and Fundamentals , 2022 .

[14]  Zhanyi Hu,et al.  PnP Problem Revisited , 2005, Journal of Mathematical Imaging and Vision.

[15]  Long Quan,et al.  Linear N-Point Camera Pose Determination , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[17]  Vincent Lepetit,et al.  An Efficient Minimal Solution for Multi-camera Motion , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[18]  Zuzana Kukelova,et al.  Automatic Generator of Minimal Problem Solvers , 2008, ECCV.

[19]  Robert M. Haralick,et al.  Analysis and solutions of the three point perspective pose estimation problem , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Bill Triggs,et al.  Camera pose and calibration from 4 or 5 known 3D points , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[21]  Lihong Zhi,et al.  A complete symbolic-numeric linear method for camera pose determination , 2003, ISSAC '03.

[22]  Michael Felsberg,et al.  Rolling shutter bundle adjustment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Marc Pollefeys,et al.  Rolling Shutter Stereo , 2013, 2013 IEEE International Conference on Computer Vision.

[24]  Michael Felsberg,et al.  Structure and motion estimation from rolling shutter video , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[25]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[26]  Per-Erik Forssén,et al.  Efficient Video Rectification and Stabilisation for Cell-Phones , 2012, International Journal of Computer Vision.

[27]  Jiri Matas,et al.  Locally Optimized RANSAC , 2003, DAGM-Symposium.

[28]  P. McAree,et al.  Using Sturm sequences to bracket real roots of polynomial equations , 1990 .

[29]  Zuzana Kukelova,et al.  Singly-Bordered Block-Diagonal Form for Minimal Problem Solvers , 2014, ACCV.

[30]  Philippe Martinet,et al.  Simultaneous Object Pose and Velocity Computation Using a Single View from a Rolling Shutter Camera , 2006, ECCV.

[31]  Chao Jia,et al.  Probabilistic 3-D motion estimation for rolling shutter video rectification from visual and inertial measurements , 2012, 2012 IEEE 14th International Workshop on Multimedia Signal Processing (MMSP).

[32]  Viktor Larsson,et al.  Efficient Solvers for Minimal Problems by Syzygy-Based Reduction , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Joel A. Hesch,et al.  A Direct Least-Squares (DLS) method for PnP , 2011, 2011 International Conference on Computer Vision.