Calcul de pose dynamique avec les caméras CMOS utilisant une acquisition séquentielle. (Dynamic pose estimation with CMOS cameras using sequential acquisition)

En informatique, la vision par ordinateur s’attache a extraire de l’information a partir de cameras. Les capteurs de celles-ci peuvent etre produits avec la technologie CMOS que nous retrouvons dans les appareils mobiles en raison de son faible cout et d’un encombrement reduit. Cette technologie permet d’acquerir rapidement l’image en exposant les lignes de l’image de maniere sequentielle. Cependant cette methode produit des deformations dans l’image s’il existe un mouvement entre la camera et la scene filmee. Cet effet est connu sous le nom de «Rolling Shutter» et de nombreuses methodes ont tente de corriger ces artefacts. Plutot que de le corriger, des travaux anterieurs ont developpe des methodes pour extraire de l’information sur le mouvement a partir de cet effet. Ces methodes reposent sur une extension de la modelisation geometrique classique des cameras pour prendre en compte l’acquisition sequentielle et le mouvement entre le capteur et la scene, considere uniforme. A partir de cette modelisation, il est possible d’etendre le calcul de pose habituel (estimation de la position et de l’orientation de la scene par rapport au capteur) pour estimer aussi les parametres du mouvement. Dans la continuite de cette demarche, nous presenterons une generalisation a des mouvements non-uniformes basee sur un lissage des derivees des parametres de mouvement. Ensuite nous presenterons une modelisation polynomiale du «Rolling Shutter» et une methode d’optimisation globale pour l’estimation de ces parametres. Correctement implemente, cela permet de realiser une mise en correspondance automatique entre le modele tridimensionnel et l’image. Pour terminer nous comparerons ces differentes methodes tant sur des donnees simulees que sur des donnees reelles et conclurons.

[1]  G. Wahba,et al.  A completely automatic french curve: fitting spline functions by cross validation , 1975 .

[2]  B. Triggs,et al.  Camera Pose Revisited -- New Linear Algorithms , 2000 .

[3]  Per-Erik Forssén,et al.  Scan rectification for structured light range sensors with rolling shutters , 2011, 2011 International Conference on Computer Vision.

[4]  M. Laurent Sums of Squares, Moment Matrices and Optimization Over Polynomials , 2009 .

[5]  Ken Shoemake,et al.  Animating rotation with quaternion curves , 1985, SIGGRAPH.

[6]  Jack J. Dongarra,et al.  A set of level 3 basic linear algebra subprograms , 1990, TOMS.

[7]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[8]  C. Vogel Non-convergence of the L-curve regularization parameter selection method , 1996 .

[9]  Y. Tsai Roger An Efficient and Accurate Camera Calibration Technique For 3D Machine Vision , 1986, CVPR 1986.

[10]  Larry S. Davis,et al.  Model-based object pose in 25 lines of code , 1992, International Journal of Computer Vision.

[11]  Nicolas Andreff,et al.  High-speed pose and velocity measurement from vision , 2008, 2008 IEEE International Conference on Robotics and Automation.

[12]  Jean B. Lasserre,et al.  Global Optimization with Polynomials and the Problem of Moments , 2000, SIAM J. Optim..

[13]  Rajiv Gupta,et al.  Linear Pushbroom Cameras , 1994, ECCV.

[14]  David W. Murray,et al.  Parallel Tracking and Mapping on a camera phone , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[15]  Kim-Chuan Toh,et al.  SDPT3 -- A Matlab Software Package for Semidefinite Programming , 1996 .

[16]  Brian Borchers,et al.  Implementation of a primal–dual method for SDP on a shared memory parallel architecture , 2007, Comput. Optim. Appl..

[17]  Jean B. Lasserre,et al.  A semidefinite programming approach to the generalized problem of moments , 2007, Math. Program..

[18]  R. Fletcher Practical Methods of Optimization , 1988 .

[19]  Nikos Paragios,et al.  Handbook of Mathematical Models in Computer Vision , 2005 .

[20]  D. Shanno Conditioning of Quasi-Newton Methods for Function Minimization , 1970 .

[21]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[22]  G. M.,et al.  Projective Geometry , 1938, Nature.

[23]  Tae-Chan Kim,et al.  CIS video panoramic image , 2012, 2012 IEEE 16th International Symposium on Consumer Electronics.

[24]  S. Axler Linear Algebra Done Right , 1995, Undergraduate Texts in Mathematics.

[25]  Jorge J. Moré,et al.  The Levenberg-Marquardt algo-rithm: Implementation and theory , 1977 .

[26]  Peter J Seiler,et al.  SOSTOOLS: Sum of squares optimization toolbox for MATLAB , 2002 .

[27]  Sergei V. Pereverzev,et al.  On the generalized discrepancy principle for Tikhonov regularization in Hilbert scales , 2010 .

[28]  Edouard Laroche,et al.  Dynamical models for position measurement with global shutter and rolling shutter cameras , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[29]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[30]  Per-Erik Forssén,et al.  Rectifying rolling shutter video from hand-held devices , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Leon S. Lasdon,et al.  Experiments with successive quadratic programming algorithms , 1988 .

[32]  Derek Bradley,et al.  Synchronization and rolling shutter compensation for consumer video camera arrays , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[33]  Pablo A. Parrilo,et al.  Semidefinite programming relaxations for semialgebraic problems , 2003, Math. Program..

[34]  Ki-Sang Hong,et al.  CMOS Digital Image Stabilization , 2007, IEEE Transactions on Consumer Electronics.

[35]  Jack J. Dongarra,et al.  Automated empirical optimizations of software and the ATLAS project , 2001, Parallel Comput..

[36]  D. Marquardt An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[37]  Ed Anderson,et al.  LAPACK Users' Guide , 1995 .

[38]  Nicolas Andreff,et al.  Kinematics from Lines in a Single Rolling Shutter Image , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Tom Drummond,et al.  Machine Learning for High-Speed Corner Detection , 2006, ECCV.

[40]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[41]  Didier Henrion,et al.  GloptiPoly 3: moments, optimization and semidefinite programming , 2007, Optim. Methods Softw..

[42]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[43]  Thomas Bonesky Morozov's discrepancy principle and Tikhonov-type functionals , 2008 .

[44]  A. Bartoli,et al.  A Generic Rolling Shutter Camera Model and its Application to Dynamic Pose Estimation , 2010 .

[45]  Ki-Sang Hong,et al.  Affine Motion Based CMOS Distortion Analysis and CMOS Digital Image Stabilization , 2007, IEEE Transactions on Consumer Electronics.

[46]  Martin Byröd,et al.  Fast and Stable Polynomial Equation Solving and Its Application to Computer Vision , 2009, International Journal of Computer Vision.

[47]  M. J. D. Powell,et al.  Variable Metric Methods for Constrained Optimization , 1982, ISMP.

[48]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[49]  Bill Triggs,et al.  Autocalibration from Planar Scenes , 1998, ECCV.

[50]  Michael Felsberg,et al.  Structure and motion estimation from rolling shutter video , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[51]  S. Shankar Sastry,et al.  Geometric Models of Rolling-Shutter Cameras , 2005, ArXiv.

[52]  Janne Heikkilä,et al.  A four-step camera calibration procedure with implicit image correction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[53]  ProblemsPer Christian HansenDepartment The L-curve and its use in the numerical treatment of inverse problems , 2000 .

[54]  Stephen J. Maybank,et al.  On plane-based camera calibration: A general algorithm, singularities, applications , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[55]  Per Christian Hansen,et al.  Analysis of Discrete Ill-Posed Problems by Means of the L-Curve , 1992, SIAM Rev..

[56]  François Berry,et al.  Structure and kinematics triangulation with a rolling shutter stereo rig , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[57]  Tomas Pajdla,et al.  Geometry of Two-Slit Camera , 2002 .

[58]  Bill Triggs,et al.  Autocalibration and the absolute quadric , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[59]  B. Borchers CSDP, A C library for semidefinite programming , 1999 .

[60]  Larry Nazareth,et al.  A family of variable metric updates , 1977, Math. Program..

[61]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[62]  Michael Felsberg,et al.  Rolling shutter bundle adjustment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[63]  Ki-Sang Hong,et al.  A fast CIS still image stabilization method without parallax and moving object problems , 2008, IEEE Transactions on Consumer Electronics.

[64]  Jean B. Lasserre,et al.  Moments and sums of squares for polynomial optimization and related problems , 2009, J. Glob. Optim..

[65]  Philippe Martinet,et al.  3D pose and velocity visual tracking based on sequential region of interest acquisition , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[66]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[67]  Richard H. Middleton,et al.  Rolling Shutter Image Compensation , 2006, RoboCup.

[68]  Per-Erik Forssén,et al.  Efficient Video Rectification and Stabilisation for Cell-Phones , 2012, International Journal of Computer Vision.

[69]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[70]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[71]  Chong-Min Kyung,et al.  Suppressing rolling-shutter distortion of CMOS image sensors by motion vector detection , 2008, IEEE Transactions on Consumer Electronics.

[72]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[73]  Long Quan,et al.  Linear N-Point Camera Pose Determination , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[74]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[75]  Ki-Sang Hong,et al.  Digital Video Stabilization Algorithm for CMOS Image Sensor , 2006, 2006 International Conference on Image Processing.

[76]  Zuzana Kukelova,et al.  Automatic Generator of Minimal Problem Solvers , 2008, ECCV.

[77]  V. Lepetit,et al.  EPnP: An Accurate O(n) Solution to the PnP Problem , 2009, International Journal of Computer Vision.

[78]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[79]  Leonard McMillan,et al.  General Linear Cameras , 2004, ECCV.

[80]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[81]  John W. Auer,et al.  Linear algebra with applications , 1996 .

[82]  Terrance E. Boult,et al.  Correcting rolling-shutter distortion of CMOS sensors using facial feature detection , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[83]  C. G. Broyden The Convergence of a Class of Double-rank Minimization Algorithms 2. The New Algorithm , 1970 .

[84]  Ludovic Magerand,et al.  Global Optimization of Object Pose and Motion from a Single Rolling Shutter Image with Automatic 2D-3D Matching , 2012, ECCV.

[85]  D. C. Brown,et al.  Lens distortion for close-range photogrammetry , 1986 .

[86]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[87]  Gene H. Golub,et al.  Singular value decomposition and least squares solutions , 1970, Milestones in Matrix Computation.

[88]  Nassir Navab,et al.  L-Tangent Norm: A Low Computational Cost Criterion for Choosing Regularization Weights and its Use for Range Surface Reconstruction , 2008 .

[89]  Kenneth Levenberg A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[90]  Philippe Martinet,et al.  Simultaneous Object Pose and Velocity Computation Using a Single View from a Rolling Shutter Camera , 2006, ECCV.

[91]  R. Fletcher,et al.  A New Approach to Variable Metric Algorithms , 1970, Comput. J..