Inertial vision pose estimation using non-linear observers

This thesis considers the problem of obtaining a high quality estimate of pose (position and orientation) from a combination of inertial and vision measurements using low cost sensors. The novelty of this work is in using non-linear observers designed on the Lie group SE(3). This approach results in robust estimators with strong local and almost-global stability results and straightforward gain tuning. A range of sensor models and observer designs are investigated, including • Creating a cascaded pose observer design using component observers for orientation and position, combining pose measurements reconstructed from vision with inertial measurements of angular velocity and linear acceleration. • Combining inertial measurement of angular and linear velocity with pose measurements reconstructed from vision, and designing an observer using a decomposition of SE(3) into separate orientation and position components. • Considering the case where the inertial measurements of angular and linear velocity are corrupted by slowly time varying biases and developing an observer for both pose and velocity measurement bias directly on SE(3)× se(3). • Eliminating the need for a pose measurement reconstruction by designing an observer operating directly on the measurements of landmark bearing from a vision sensor. The bearing measurements are combined with unbiased measurements of linear and angular velocity measurements to obtain an observer for pose evolving on SE(3). Throughout these works, particular attention is given to designing observers suitable for implementation with multiple independent measurement devices. Care is given to avoid coupling of independent measurement noise processes in estimator dynamics. Further, the final observers proposed are suitable for a multi-rate asynchronous implementation, making timely use of measurements from sensors whose measurement rates may differ by over an order of magnitude.

[1]  Xinkai Chen,et al.  A new state observer for perspective systems , 2002, IEEE Trans. Autom. Control..

[2]  Paul E. I. Pounds,et al.  Design, construction and control of a large quadrotor micro air vehicle , 2007 .

[3]  Stergios I. Roumeliotis,et al.  A Multi-State Constraint Kalman Filter for Vision-aided Inertial Navigation , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[4]  H.F. Durrant-Whyte,et al.  A new approach for filtering nonlinear systems , 1995, Proceedings of 1995 American Control Conference - ACC'95.

[5]  Robert E. Mahony,et al.  Adaptive filtering and image based visual servo control of a ducted fan flying robot , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[6]  Jorge Lobo,et al.  Integration of vision and inertial sensing , 2007 .

[7]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[8]  Tao Wu,et al.  Observability analysis of rotation estimation by fusing inertial and line-based visual information: A revisit , 2006, Autom..

[9]  S. Salcudean A globally convergent angular velocity observer for rigid body motion , 1991 .

[10]  Masoud Salehi,et al.  Communication Systems Engineering , 1994 .

[11]  R. G. Brown Integrated Navigation Systems and Kalman Filtering: A Perspective , 1972 .

[12]  Stephen M. Rock,et al.  Relative position sensing by fusing monocular vision and inertial rate sensors , 2003 .

[13]  João M. F. Xavier,et al.  Maximum Likelihood Attitude and Position Estimation from Pseudo-Range Measurements using Geometric Descent Optimization , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[14]  Peter I. Corke,et al.  Low-cost flight control system for a small autonomous helicopter , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[15]  Robert E. Mahony,et al.  Attitude estimation on SO[3] based on direct inertial measurements , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[16]  R. E. Kalman,et al.  New Results in Linear Filtering and Prediction Theory , 1961 .

[17]  T. Hamel,et al.  A practical Visual Servo Control for a Unmanned Aerial Vehicle , 2008, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[18]  Jeffrey K. Uhlmann,et al.  Reduced sigma point filters for the propagation of means and covariances through nonlinear transformations , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[19]  Bijoy K. Ghosh,et al.  Pose estimation using line-based dynamic vision and inertial sensors , 2003, IEEE Trans. Autom. Control..

[20]  Philippe Martin,et al.  Design and Implementation of a Low-Cost Aided Attitude and Heading Reference System , 2008 .

[21]  Philippe Martin,et al.  A general symmetry-preserving observer for aided attitude heading reference systems , 2008, 2008 47th IEEE Conference on Decision and Control.

[22]  Gaurav S. Sukhatme,et al.  A tale of two helicopters , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[23]  Rita Cunha,et al.  Vision-based control for rigid body stabilization , 2007, 2007 46th IEEE Conference on Decision and Control.

[24]  M. Shuster,et al.  Three-axis attitude determination from vector observations , 1981 .

[25]  Rita Cunha,et al.  Stability of a nonlinear attitude observer on SO(3) with nonideal angular velocity measurements , 2009, 2009 European Control Conference (ECC).

[26]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[27]  P. Rouchon,et al.  A non-linear symmetry-preserving observer for velocity-aided inertial navigation , 2006, 2006 American Control Conference.

[28]  E. J. Lefferts,et al.  Kalman Filtering for Spacecraft Attitude Estimation , 1982 .

[29]  Omead Amidi,et al.  An autonomous vision-guided helicopter , 1996 .

[30]  J. Crassidis,et al.  Particle Filtering for Sequential Spacecraft Attitude Estimation , 2004 .

[31]  Jonghoon Park,et al.  Geometric integration on Euclidean group with application to articulated multibody systems , 2005, IEEE Transactions on Robotics.

[32]  Xiaoming Hu,et al.  Observers for systems with implicit output , 2000, IEEE Trans. Autom. Control..

[33]  Rudolph van der Merwe,et al.  The unscented Kalman filter for nonlinear estimation , 2000, Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No.00EX373).

[34]  Daniel D. Lee,et al.  Rao-Blackwellized particle filtering for 6-DOF estimation of attitude and position via GPS and inertial sensors , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[35]  P. Perona,et al.  Motion estimation via dynamic vision , 1996, IEEE Trans. Autom. Control..

[36]  Peter I. Corke An inertial and visual sensing system for a small autonomous helicopter , 2004 .

[37]  Eric W. Frew,et al.  Relative position estimation for AUVs by fusing bearing and inertial rate sensor measurements , 2002, OCEANS '02 MTS/IEEE.

[38]  Lorenzo Marconi,et al.  Control and Observer Design for Nonlinear Finite and Infinite Dimensional Systems , 2005 .

[39]  Fredrik Gustafsson,et al.  Particle filters for positioning, navigation, and tracking , 2002, IEEE Trans. Signal Process..

[40]  Robert M. Sanner,et al.  A Coupled Nonlinear Spacecraft Attitude Controller and Observer with an Unknown Gyro Misalignment and Gyro Bias , 2002 .

[41]  João Pedro Hespanha,et al.  Minimum-energy state estimation for systems with perspective outputs , 2006, IEEE Transactions on Automatic Control.

[42]  Peter Corke,et al.  High-Performance Visual Closed-Loop Robot Control , 1994 .

[43]  Robert E. Mahony,et al.  Robust Nonlinear Fusion of Inertial and Visual Data for position, velocity and attitude estimation of UAV , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[44]  Simon J. Julier,et al.  The scaled unscented transformation , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[45]  Markus Vincze,et al.  Multi-rate fusion with vision and inertial sensors , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[46]  Kostas E. Bekris,et al.  Evaluation of algorithms for bearing-only SLAM , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[47]  Silvere Bonnabel,et al.  Left-invariant extended Kalman filter and attitude estimation , 2007, 2007 46th IEEE Conference on Decision and Control.

[48]  P. Crouch,et al.  Numerical integration of ordinary differential equations on manifolds , 1993 .

[49]  B. Ghosh,et al.  Visually guided ranging from observations of points, lines and curves via an identifier based nonlinear observer , 1995 .

[50]  Markus Vincze,et al.  Fast Ego-motion Estimation with Multi-rate Fusion of Inertial and Vision , 2007, Int. J. Robotics Res..

[51]  P. Absil Optimization on manifolds , 2007 .

[52]  Thor I. Fossen,et al.  A nonlinear observer for GPS and INS integration , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[53]  Huamin Jia,et al.  A Review of Multisensor Fusion Methodologies for Aircraft Navigation Systems , 2005 .

[54]  Son-Goo Kim,et al.  Kalman filtering for relative spacecraft attitude and position estimation , 2005 .

[55]  Rita Cunha,et al.  Output-feedback control for stabilization on SE(3) , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[56]  C. Scovel,et al.  Symplectic integration of Hamiltonian systems , 1990 .

[57]  J. Vasconcelos,et al.  A Nonlinear Observer for Rigid Body Attitude Estimation using Vector Observations , 2008 .

[58]  Isaac Kaminer,et al.  Navigation system design using time-varying complementary filters , 1999 .

[59]  Bijoy K. Ghosh,et al.  Nonlinear observers for perspective time-invariant linear systems , 2004, Autom..

[60]  Mazyar Mirrahimi,et al.  Observer-based Hamiltonian identification for quantum systems , 2007, Autom..

[61]  Mrdjan Jankovic,et al.  Some Problems In Perspective System Theory And Its Application To Machine Vision , 1992, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems.

[62]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[63]  J. Huang,et al.  Curse of dimensionality and particle filters , 2003, 2003 IEEE Aerospace Conference Proceedings (Cat. No.03TH8652).

[64]  F. Markley,et al.  Nonlinear Attitude Filtering Methods , 2005 .

[65]  Phillip J. McKerrow,et al.  Introduction to robotics , 1991 .

[66]  Pierre Rouchon,et al.  An intrinsic observer for a class of Lagrangian systems , 2003, IEEE Trans. Autom. Control..

[67]  F. Callier Introduction to Mathematical Systems Theory: A Behavioural Approach, by Jan Willem Polderman and Jan C. Willems, Springer, New York, NY, 1998, Texts in Applied Mathematics Vol. 26, 424pp. , 2002 .

[68]  Eduardo Sontag,et al.  On characterizations of the input-to-state stability property , 1995 .

[69]  Levent Tunçel,et al.  Optimization algorithms on matrix manifolds , 2009, Math. Comput..

[70]  S. Bonnabel,et al.  Symmetry-preserving nudging: theory and application to a shallow water model , 2008 .

[71]  R. D. Vogelaere,et al.  Methods of Integration which Preserve the Contact Transformation Property of the Hamilton Equations , 1956 .

[72]  Philippe Martin,et al.  Non-Linear Symmetry-Preserving Observers on Lie Groups , 2007, IEEE Transactions on Automatic Control.

[73]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[74]  Philippe Martin,et al.  Invariant observers for attitude and heading estimation from low-cost inertial and magnetic sensors , 2007, 2007 46th IEEE Conference on Decision and Control.

[75]  João M. F. Xavier,et al.  Estimation of Attitude and Position from Range-Only Measurements using Geometric Descent Optimization on the Special Euclidean Group , 2006, 2006 9th International Conference on Information Fusion.

[76]  J.P. Hespanha,et al.  State Estimation for Systems with Implicit Outputs for the Integration of Vision and Inertial Sensors , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[77]  Xiaoying Kong INS algorithm using quaternion model for low cost IMU , 2004, Robotics Auton. Syst..

[78]  F. Park Computational aspects of the product-of-exponentials formula for robot kinematics , 1994, IEEE Trans. Autom. Control..

[79]  Wen-Hong Zhu,et al.  Velocity Estimation by Using Position and Acceleration Sensors , 2007, IEEE Transactions on Industrial Electronics.

[80]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[81]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[82]  J. Hespanha State estimation and control for systems with perspective outputs , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[83]  Ziyang Meng,et al.  A brief survey of the deterministic solution for satellite attitude estimation , 2008, International Symposium on Instrumentation and Control Technology.

[84]  M. Pachter,et al.  Bearings-only measurements for INS aiding: the three-dimensional case , 2004, Proceedings of the 2004 American Control Conference.

[85]  Lee E. Weiss,et al.  Dynamic visual servo control of robots: An adaptive image-based approach , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[86]  Eduardo Sontag Smooth stabilization implies coprime factorization , 1989, IEEE Transactions on Automatic Control.

[87]  S. Bhat,et al.  A topological obstruction to continuous global stabilization of rotational motion and the unwinding phenomenon , 2000 .

[88]  R.E. Ziemer,et al.  Digital and analog communication systems , 1981, Proceedings of the IEEE.

[89]  Alain Sarlette,et al.  Coordinated Motion Design on Lie Groups , 2008, IEEE Transactions on Automatic Control.

[90]  Philippe Martin,et al.  An Invariant Observer for Earth-Velocity-Aided Attitude Heading Reference Systems , 2008 .

[91]  Silvere Bonnabel,et al.  Groupe de Lie et observateur non-linéaire , 2006 .

[92]  Taeyoung Lee,et al.  Global optimal attitude estimation using uncertainty ellipsoids , 2006, Syst. Control. Lett..

[93]  Gregory D. Hager,et al.  Robust Vision for Vision-Based Control of Motion , 1999 .

[94]  Jeffrey K. Uhlmann,et al.  New extension of the Kalman filter to nonlinear systems , 1997, Defense, Security, and Sensing.

[95]  Alain Sarlette,et al.  Coordination on Lie groups , 2008, 2008 47th IEEE Conference on Decision and Control.

[96]  Panagiotis D. Christofides,et al.  Control of Nonlinear and Hybrid Process Systems: Designs for Uncertainty, Constraints and Time-Delays , 2005 .

[97]  Markus Vincze,et al.  On multi-rate fusion for non-linear sampled-data systems: Application to a 6D tracking system , 2008, Robotics Auton. Syst..

[98]  Charles R. Johnson,et al.  Topics in Matrix Analysis , 1991 .

[99]  Radu Horaud,et al.  An analytic solution for the perspective 4-point problem , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[100]  Gordon Wyeth,et al.  Sensing and Control for a Small-Size Helicopter , 2002, ISER.

[101]  Weiping Li,et al.  Applied Nonlinear Control , 1991 .

[102]  Robert E. Mahony,et al.  Nonlinear Complementary Filters on the Special Orthogonal Group , 2008, IEEE Transactions on Automatic Control.

[103]  Philippe Martin,et al.  Design and Implementation of a Low-cost Attitude and Heading Nonlinear Estimator , 2008, ICINCO-SPSMC.

[104]  Philippe Martin,et al.  Symmetry-Preserving Observers , 2006, IEEE Transactions on Automatic Control.

[105]  David L. Mills,et al.  Internet Engineering Task Force (ietf) Network Time Protocol Version 4: Protocol and Algorithms Specification , 2010 .

[106]  Petros G. Voulgaris,et al.  On optimal ℓ∞ to ℓ∞ filtering , 1995, Autom..

[107]  Stephen M. Rock,et al.  RELATIVE POSITION ESTIMATION FOR INTERVENTION-CAPABLE AUVS BY FUSING VISION AND INERTIAL MEASUREMENTS , .

[108]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[109]  Hugh Durrant-Whyte,et al.  Simultaneous Localisation and Mapping ( SLAM ) : Part I The Essential Algorithms , 2006 .

[110]  Rita Cunha,et al.  Landmark based nonlinear observer for rigid body attitude and position estimation , 2007, 2007 46th IEEE Conference on Decision and Control.

[111]  W. Boothby An introduction to differentiable manifolds and Riemannian geometry , 1975 .

[112]  Philip David,et al.  SoftPOSIT: An Algorithm for Registration of 3D Models to Noisy Perspective Images Combining Softassi , 2001 .

[113]  T. Hamel,et al.  Complementary filter design on the special orthogonal group SO(3) , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[114]  A. Vedaldi,et al.  Inertial Structure From Motion with Autocalibration , 2007 .