Fusion de données visuo-inertielles pour l'estimation de pose et l'autocalibrage

Les systemes multi-capteurs exploitent les complementarites des differentes sources sensorielles. Par exemple, le capteur visuo-inertiel permet d’estimer la pose a haute frequence et avec une grande precision. Les methodes de vision mesurent la pose a basse frequence mais limitent la derive causee par l’integration des donnees inertielles. Les centrales inertielles mesurent des increments du deplacement a haute frequence, ce que permet d’initialiser la vision et de compenser la perte momentanee de celle-ci. Cette these analyse deux aspects du probleme. Premierement, nous etudions les methodes visuelles directes pour l’estimation de pose, et proposons une nouvelle technique basee sur la correlation entre des images et la ponderation des regions et des pixels, avec une optimisation inspiree de la methode de Newton. Notre technique estime la pose meme en presence des changements d’illumination extremes. Deuxiemement, nous etudions la fusion des donnees a partir de la theorie de la commande. Nos resultats principaux concernent le developpement d’observateurs pour l’estimation de pose, biais IMU et l’autocalibrage. Nous analysons la dynamique de rotation d’un point de vue non lineaire, et fournissons des observateurs stables dans le groupe des matrices de rotation. Par ailleurs, nous analysons la dynamique de translation en tant que systeme lineaire variant dans le temps, et proposons des conditions d’observabilite uniforme. Les analyses d’observabilite nous permettent de demontrer la stabilite uniforme des observateurs proposes. La methode visuelle et les observateurs sont testes et compares aux methodes classiques avec des simulations et de vraies donnees visuo-inertielles.

[1]  R. Kálmán On the general theory of control systems , 1959 .

[2]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[3]  R. E. Kalman,et al.  New Results in Linear Filtering and Prediction Theory , 1961 .

[4]  H. D. Black,et al.  A passive system for determining the attitude of a satellite , 1964 .

[5]  C. Langenhop On the stabilization of linear systems , 1964 .

[6]  Y. Ho,et al.  A Bayesian approach to problems in stochastic estimation and control , 1964 .

[7]  D. Luenberger Observing the State of a Linear System , 1964, IEEE Transactions on Military Electronics.

[8]  G. Wahba A Least Squares Estimate of Satellite Attitude , 1965 .

[9]  D. Luenberger Observers for multivariable systems , 1966 .

[10]  P. B. Davenport A vector approach to the algebra of rotations with applications , 1968 .

[11]  B. Anderson,et al.  NEW RESULTS IN LINEAR SYSTEM STABILITY , 1969 .

[12]  A. Krener,et al.  Nonlinear controllability and observability , 1977 .

[13]  James F. Blinn,et al.  Models of light reflection for computer synthesized pictures , 1977, SIGGRAPH.

[14]  M. D. Shuster Approximate algorithms for fast optimal attitude computation , 1978 .

[15]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[16]  C. Draper Origins of inertial navigation , 1981 .

[17]  S. Liberty,et al.  Linear Systems , 2010, Scientific Parallel Computing.

[18]  E. J. Lefferts,et al.  Kalman Filtering for Spacecraft Attitude Estimation , 1982 .

[19]  Robert L. Cook,et al.  A Reflectance Model for Computer Graphics , 1987, TOGS.

[20]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[21]  Arthur J. Krener,et al.  Linearization by output injection and nonlinear observers , 1983 .

[22]  H. W. Sorenson,et al.  Kalman filtering : theory and application , 1985 .

[23]  J. Gauthier,et al.  Observability and observers for non-linear systems , 1986, 1986 25th IEEE Conference on Decision and Control.

[24]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[25]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[26]  Olivier Faugeras,et al.  Motion and Structure from Motion in a piecewise Planar Environment , 1988, Int. J. Pattern Recognit. Artif. Intell..

[27]  G. Bornard,et al.  Regularly persistent observers for bilinear systems , 1989 .

[28]  Roger Y. Tsai,et al.  A new technique for fully autonomous and efficient 3D robotics hand/eye calibration , 1988, IEEE Trans. Robotics Autom..

[29]  Hervé Reinhard Equations différentielles : fondements et applications , 1989 .

[30]  H. Hammouri,et al.  Observer synthesis for state-affine systems , 1990, 29th IEEE Conference on Decision and Control.

[31]  Arjan van der Schaft,et al.  Non-linear dynamical control systems , 1990 .

[32]  Eduardo D. Sontag,et al.  Mathematical Control Theory: Deterministic Finite Dimensional Systems , 1990 .

[33]  Olivier Faugeras,et al.  Computation of inertial information on a Robot , 1991 .

[34]  S. Salcudean A globally convergent angular velocity observer for rigid body motion , 1991 .

[35]  J. Gauthier,et al.  Global time-varying linearization up to output injection , 1992 .

[36]  H. Khalil,et al.  Output feedback stabilization of fully linearizable systems , 1992 .

[37]  F. W. Cathey,et al.  The iterated Kalman filter update as a Gauss-Newton method , 1993, IEEE Trans. Autom. Control..

[38]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[39]  P. Pardalos,et al.  Handbook of global optimization , 1995 .

[40]  Chi-Tsong Chen,et al.  Linear System Theory and Design , 1995 .

[41]  A. Hanson,et al.  Scaled Euclidean 3D reconstruction based on externally uncalibrated cameras , 1995, Proceedings of International Symposium on Computer Vision - ISCV.

[42]  Henk Nijmeijer,et al.  An observer looks at synchronization , 1997 .

[43]  Zhengyou Zhang,et al.  Parameter estimation techniques: a tutorial with application to conic fitting , 1997, Image Vis. Comput..

[44]  N. Ayache,et al.  Multimodal Image Registration by Maximization of the Correlation Ratio , 1998 .

[45]  P. Anandan,et al.  Robust multi-sensor image alignment , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[46]  The XVision system : a general-purpose substrate for portable real-time vision applications , 1998 .

[47]  Gregory D. Hager,et al.  Efficient Region Tracking With Parametric Models of Geometry and Illumination , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[48]  Nicholas Ayache,et al.  The Correlation Ratio as a New Similarity Measure for Multimodal Image Registration , 1998, MICCAI.

[49]  Ronald Azuma,et al.  A motion-stabilized outdoor augmented reality system , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[50]  P. Rouchon,et al.  Invariant tracking and stabilization: problem formulation and examples , 1999 .

[51]  Sukhan Lee,et al.  Micromachined inertial sensors , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[52]  Éric Marchand,et al.  VISP: a software environment for eye-in-hand visual servoing , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[53]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[54]  Johann Borenstein,et al.  Precision calibration of fiber-optics gyroscopes for mobile robot navigation , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[55]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[56]  Hugh F. Durrant-Whyte,et al.  A new method for the nonlinear transformation of means and covariances in filters and estimators , 2000, IEEE Trans. Autom. Control..

[57]  Nando de Freitas,et al.  Rao-Blackwellised Particle Filtering for Dynamic Bayesian Networks , 2000, UAI.

[58]  Thor I. Fossen,et al.  A nonlinear observer for GPS and INS integration , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[59]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[60]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[61]  Simon Baker,et al.  Equivalence and efficiency of image alignment algorithms , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[62]  Jean-Paul Gauthier,et al.  HIGH-GAIN AND NON-HIGH-GAIN OBSERVERS FOR NONLINEAR SYSTEMS , 2002 .

[63]  S. Shankar Sastry,et al.  An Invitation to 3-D Vision: From Images to Geometric Models , 2003 .

[64]  Jorge Dias,et al.  Vision and Inertial Sensor Cooperation Using Gravity as a Vertical Reference , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[65]  Eric Foxlin,et al.  Miniaturization, calibration & accuracy evaluation of a hybrid self-tracker , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[66]  Robert M. Sanner,et al.  A coupled nonlinear spacecraft attitude controller and observer with an unknown constant gyro bias and gyro noise , 2003, IEEE Trans. Autom. Control..

[67]  Bijoy K. Ghosh,et al.  Pose estimation using line-based dynamic vision and inertial sensors , 2003, IEEE Trans. Autom. Control..

[68]  Markus Vincze,et al.  Multi-rate fusion with vision and inertial sensors , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[69]  Jeffrey K. Uhlmann,et al.  Unscented filtering and nonlinear estimation , 2004, Proceedings of the IEEE.

[70]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[71]  Ezio Malis,et al.  Improving vision-based control using efficient second-order minimization techniques , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[72]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[73]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[74]  M. C. VanDyke,et al.  UNSCENTED KALMAN FILTERING FOR SPACECRAFT ATTITUDE STATE AND PARAMETER ESTIMATION , 2004 .

[75]  Amir Averbuch,et al.  Fast motion estimation using bidirectional gradient methods , 2004, IEEE Transactions on Image Processing.

[76]  F. Daum Nonlinear filters: beyond the Kalman filter , 2005, IEEE Aerospace and Electronic Systems Magazine.

[77]  A. Pinz,et al.  Calibration of Hybrid Vision / Inertial Tracking Systems * , 2005 .

[78]  T. Hamel,et al.  Complementary filter design on the special orthogonal group SO(3) , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[79]  Jordan M. Berg,et al.  Intrinsic Observer-Based Stabilization for Simple Mechanical Systems on Lie Groups , 2005, SIAM J. Control. Optim..

[80]  Héctor J. Sussmann,et al.  Single-input observability of continuous-time systems , 1978, Mathematical systems theory.

[81]  Baba C. Vemuri,et al.  Non-Rigid Multi-Modal Image Registration Using Cross-Cumulative Residual Entropy , 2007, International Journal of Computer Vision.

[82]  J.L. Crassidis,et al.  Sigma-point Kalman filtering for integrated GPS and inertial navigation , 2005, IEEE Transactions on Aerospace and Electronic Systems.

[83]  Robert E. Mahony,et al.  Attitude estimation on SO[3] based on direct inertial measurements , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[84]  Domenico Campolo,et al.  Inertial/Magnetic Sensors Based Orientation Tracking on the Group of Rigid Body Rotations with Application to Wearable Devices , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[85]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[86]  A. Sanyal Optimal Attitude Estimation and Filtering Without Using Local Coordinates Part I: Uncontrolled and Deterministic Attitude Dynamics , 2005, 2006 American Control Conference.

[87]  Dan Simon,et al.  Optimal State Estimation: Kalman, H∞, and Nonlinear Approaches , 2006 .

[88]  Ben Himane,et al.  Vers une approche unifiée pour le suivi temps réel et l'asservissement visuel , 2006 .

[89]  Gyula Mester,et al.  Motion Control of Wheeled Mobile Robots , 2006 .

[90]  Rita Cunha,et al.  Landmark based nonlinear observer for rigid body attitude and position estimation , 2007, 2007 46th IEEE Conference on Decision and Control.

[91]  E. Malis,et al.  Deeper understanding of the homography decomposition for vision-based control , 2007 .

[92]  K. V. Arya,et al.  Image registration using robust M-estimators , 2007, Pattern Recognit. Lett..

[93]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[94]  M. Moallem,et al.  Attitude estimation and stabilization of a rigid body using low-cost sensors , 2007, 2007 46th IEEE Conference on Decision and Control.

[95]  Selim Benhimane,et al.  Homography-based 2D Visual Tracking and Servoing , 2007, Int. J. Robotics Res..

[96]  Vincent Lepetit,et al.  Fast Keypoint Recognition in Ten Lines of Code , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[97]  Peter Corke,et al.  An Introduction to Inertial and Visual Sensing , 2007, Int. J. Robotics Res..

[98]  A. Vedaldi,et al.  Inertial Structure From Motion with Autocalibration , 2007 .

[99]  John L. Crassidis,et al.  Survey of nonlinear attitude estimation methods , 2007 .

[100]  Robert E. Mahony,et al.  Robust Nonlinear Fusion of Inertial and Visual Data for position, velocity and attitude estimation of UAV , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[101]  Jorge Dias,et al.  Relative Pose Calibration Between Visual and Inertial Sensors , 2007, Int. J. Robotics Res..

[102]  G. Besançon An Overview on Observer Tools for Nonlinear Systems , 2007 .

[103]  R. Mahony,et al.  Complementary filter design on the Special Euclidean group SE(3) , 2007, 2007 European Control Conference (ECC).

[104]  Vincent Gay-Bellile,et al.  Direct Estimation of Nonrigid Registrations with Image-Based Self-Occlusion Reasoning , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[105]  Philippe Martin,et al.  An Invariant Observer for Earth-Velocity-Aided Attitude Heading Reference Systems , 2008 .

[106]  Philippe Martin,et al.  Symmetry-Preserving Observers , 2006, IEEE Transactions on Automatic Control.

[107]  Patrick Rives,et al.  An Efficient Direct Approach to Visual SLAM , 2008, IEEE Transactions on Robotics.

[108]  Carlos Silvestre,et al.  A Nonlinear GPS/IMU based observer for rigid body attitude and position estimation , 2008, 2008 47th IEEE Conference on Decision and Control.

[109]  Adrien Bartoli,et al.  Groupwise Geometric and Photometric Direct Image Registration , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[110]  J. Vasconcelos,et al.  A Nonlinear Observer for Rigid Body Attitude Estimation using Vector Observations , 2008 .

[111]  Georgios D. Evangelidis,et al.  Parametric Image Alignment Using Enhanced Correlation Coefficient Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[112]  P. Roth,et al.  SURVEY OF APPEARANCE-BASED METHODS FOR OBJECT RECOGNITION , 2008 .

[113]  Robert E. Mahony,et al.  Nonlinear Complementary Filters on the Special Orthogonal Group , 2008, IEEE Transactions on Automatic Control.

[114]  Yannick Berthoumieu,et al.  The Bi-directional Framework for Unifying Parametric Image Alignment Approaches , 2008, ECCV.

[115]  Michael R. M. Jenkin,et al.  Inertial Sensors, GPS, and Odometry , 2008, Springer Handbook of Robotics.

[116]  Robert E. Mahony,et al.  A nonlinear observer for 6 DOF pose estimation from inertial and bearing measurements , 2009, 2009 IEEE International Conference on Robotics and Automation.

[117]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[118]  Alberto Olivares,et al.  High-efficiency low-cost accelerometer-aided gyroscope calibration , 2009, 2009 International Conference on Test and Measurement.

[119]  Carlos Silvestre,et al.  Sensor-based complementary globally asymptotically stable filters for attitude estimation , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[120]  Nassir Navab,et al.  A dataset and evaluation methodology for template-based tracking algorithms , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[121]  Philippe Martin,et al.  Non-Linear Symmetry-Preserving Observers on Lie Groups , 2007, IEEE Transactions on Automatic Control.

[122]  Mark R. Pickering,et al.  A new multi-modal similarity measure for fast gradient-based 2D-3D image registration , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[123]  Minh-Duc Hua Contributions to the automatic control of aerial vehicles , 2009 .

[124]  Philippe Martin,et al.  Invariant Extended Kalman Filter: theory and application to a velocity-aided attitude estimation problem , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[125]  Minh-Duc Hua Attitude observers for accelerated rigid bodies based on GPS and INS measurements , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[126]  Tal Arbel,et al.  Generalizing Inverse Compositional and ESM Image Alignment , 2010, International Journal of Computer Vision.

[127]  Geraldo F. Silveira,et al.  Unified Direct Visual Tracking of Rigid and Deformable Surfaces Under Generic Illumination Changes in Grayscale and Color Images , 2010, International Journal of Computer Vision.

[128]  Éric Marchand,et al.  Accurate real-time tracking using mutual information , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[129]  Éric Marchand,et al.  Improving monocular plane-based SLAM with inertial measures , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[130]  Patrick Rives,et al.  A spherical robot-centered representation for urban navigation , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[131]  Philippe Martin,et al.  Design and implementation of a low-cost observer-based attitude and heading reference system , 2010 .

[132]  Thomas B. Schön,et al.  Modeling and Calibration of Inertial and Vision Sensors , 2010, Int. J. Robotics Res..

[133]  Robert E. Mahony,et al.  Gradient-Like Observers for Invariant Dynamics on a Lie Group , 2008, IEEE Transactions on Automatic Control.

[134]  Nicolas Petit,et al.  Design of a navigation filter by analysis of local observability , 2010, 49th IEEE Conference on Decision and Control (CDC).

[135]  Pascal Morin,et al.  Nonlinear filter design for pose and IMU bias estimation , 2011, 2011 IEEE International Conference on Robotics and Automation.

[136]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[137]  Stefano Soatto,et al.  Visual-inertial navigation, mapping and localization: A scalable real-time causal approach , 2011, Int. J. Robotics Res..

[138]  Gaurav S. Sukhatme,et al.  Visual-Inertial Sensor Fusion: Localization, Mapping and Sensor-to-Sensor Self-calibration , 2011, Int. J. Robotics Res..

[139]  Andrew J. Davison,et al.  DTAM: Dense tracking and mapping in real-time , 2011, 2011 International Conference on Computer Vision.

[140]  Darius Burschka,et al.  Optimization based IMU camera calibration , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[141]  Pascal Morin,et al.  A nonlinear observer approach for concurrent estimation of pose, IMU bias and camera-to-IMU rotation , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[142]  Carlos Silvestre,et al.  Accelerometer Calibration and Dynamic Bias and Gravity Estimation: Analysis, Design, and Experimental Evaluation , 2011, IEEE Transactions on Control Systems Technology.

[143]  Rita Cunha,et al.  A Nonlinear Attitude Observer Based on Active Vision and Inertial Measurements , 2011, IEEE Transactions on Robotics.

[144]  Agostino Martinelli,et al.  State Estimation Based on the Concept of Continuous Symmetry and Observability Analysis: The Case of Calibration , 2011, IEEE Transactions on Robotics.

[145]  Mrdjan J. Jankovic,et al.  Constructive Nonlinear Control , 2011 .

[146]  Robert E. Mahony,et al.  Near-Optimal Deterministic Filtering on the Rotation Group , 2011, IEEE Transactions on Automatic Control.

[147]  Russell H. Taylor,et al.  Visual tracking using the sum of conditional variance , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[148]  Maxime Meilland,et al.  Improving NCC-Based Direct Visual Tracking , 2012, ECCV.

[149]  Robert E. Mahony,et al.  Nonlinear complementary filters on the special linear group , 2012, Int. J. Control.

[150]  Agostino Martinelli,et al.  Vision and IMU Data Fusion: Closed-Form Solutions for Attitude, Speed, Absolute Scale, and Bias Determination , 2012, IEEE Transactions on Robotics.

[151]  Yilong Hao,et al.  Virtual Rate-Table Method for Characterization of Microgyroscopes , 2012, IEEE Sensors Journal.

[152]  Marko Munih,et al.  Three-Axial Accelerometer Calibration Using Kalman Filter Covariance Matrix for Online Estimation of Optimal Sensor Orientation , 2012, IEEE Transactions on Instrumentation and Measurement.

[153]  Alan F. Lynch,et al.  Invariant Observer Design for a Helicopter UAV Aided Inertial Navigation System , 2013, IEEE Transactions on Control Systems Technology.

[154]  P. Olver Nonlinear Systems , 2013 .

[155]  Robert E. Mahony,et al.  Implementation of a Nonlinear Attitude Estimator for Aerial Robotic Vehicles , 2014, IEEE Transactions on Control Systems Technology.