Real-Time Vision-Based Pose Tracking of Spacecraft in Close Range Using Geometric Curve Fitting

This article presents a new framework of real-time vision-based pose tracking for spacecraft in close range using geometric fitting of the imaged geometric primitives (GPs) on the spacecraft. At the first time instant, the tracking is initialized with the template-based pose retrieval and GP-based pose determination. At each subsequent time instant, with the pose prediction from the extended Kalman filter (EKF) as initial value, the GPs are associated with the corresponding image data, and thereby the maximum-likelihood estimation (MLE) for spacecraft pose can be obtained in real time by geometrically fitting the GP projections over the corresponding image data with generalized expectation–maximization and M-estimation. Using the MLE, the EKF generates the final pose estimation and predicts the pose at the next time instant. The basic configurations of the GPs are investigated for the stability of tracking. Sufficient experiments validate the accuracy and the real-time performance of the proposed method.

[1]  Daniele Mortari,et al.  How to avoid singularity when using Euler angles , 2005 .

[2]  Eric Joffre,et al.  RemoveDEBRIS: An in-orbit active debris removal demonstration mission , 2016 .

[3]  Éric Marchand,et al.  Tracking complex targets for space rendezvous and debris removal applications , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Manolis I. A. Lourakis,et al.  Model-based visual tracking of orbiting satellites using edges , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Simone D'Amico,et al.  Robust Model-Based Monocular Pose Initialization for Noncooperative Spacecraft Rendezvous , 2018, Journal of Spacecraft and Rockets.

[6]  Henrik I. Christensen,et al.  Robust 3D visual tracking using particle filtering on the special Euclidean group: A combined approach of keypoint and edge features , 2011, 2011 IEEE International Conference on Robotics and Automation.

[7]  Vincent Lepetit,et al.  Gradient Response Maps for Real-Time Detection of Textureless Objects , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Éric Marchand,et al.  Combining complementary edge, keypoint and color features in model-based tracking for highly dynamic scenes , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[9]  Sumant Sharma,et al.  REDUCED-DYNAMICS POSE ESTIMATION FOR NON-COOPERATIVE SPACECRAFT RENDEZVOUS USING MONOCULAR VISION , 2017 .

[10]  R. Stephenson A and V , 1962, The British journal of ophthalmology.

[11]  Francesco Branz,et al.  A relative navigation sensor for CubeSats based on LED fiducial markers , 2018 .

[12]  P. Holland,et al.  Robust regression using iteratively reweighted least-squares , 1977 .

[13]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[14]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[15]  Giorgio Panin,et al.  Vision-based localization for on-orbit servicing of a partially cooperative satellite , 2015 .

[16]  ChoiChanghyun,et al.  Robust 3D visual tracking using particle filtering on the special Euclidean group , 2012 .

[17]  Xiao Chen,et al.  A High-accuracy Extraction Algorithm of Planet Centroid Image in Deep-space Autonomous Optical Navigation , 2015, Journal of Navigation.

[18]  Ian D. Reid,et al.  PWP3D: Real-Time Segmentation and Tracking of 3D Objects , 2012, International Journal of Computer Vision.

[19]  Zhiguo Jiang,et al.  Vision-Based Pose Estimation for Textureless Space Objects by Contour Points Matching , 2018, IEEE Transactions on Aerospace and Electronic Systems.

[20]  Andrew Zisserman,et al.  Robust Object Tracking , 2001 .

[21]  Wen-Hsiang Tsai,et al.  Location estimation and trajectory prediction of moving lateral vehicle using two wheel shapes information in 2-D lateral vehicle images by 3-D computer vision techniques , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[22]  M. Pauline Baker,et al.  Computer graphics with OpenGL , 1986 .

[23]  Lei Chen,et al.  Pose estimation of large non-cooperative spacecraft based on extended PnP model , 2016, 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[24]  François Chaumette,et al.  RemoveDebris Vision-Based Navigation preliminary results , 2019 .

[25]  E. Glenn Lightsey,et al.  An On-Board Image Processing Algorithm for a Spacecraft Optical Navigation Sensor System , 2012 .

[26]  Simone D'Amico,et al.  Pose estimation for non-cooperative spacecraft rendezvous using convolutional neural networks , 2018, 2018 IEEE Aerospace Conference.

[27]  Chang Liu,et al.  Relative pose estimation for cylinder-shaped spacecrafts using single image , 2014, IEEE Transactions on Aerospace and Electronic Systems.

[28]  Bo Xu,et al.  Autonomous orbit and attitude determination for Earth satellites using images of regular-shaped ground objects , 2018 .

[29]  Anthony J. Yezzi,et al.  Robust 3D Pose Estimation and Efficient 2D Region-Based Segmentation from a 3D Shape Prior , 2008, ECCV.

[30]  Kai Li,et al.  Fast 3D Line Segment Detection From Unorganized Point Cloud , 2019, ArXiv.

[31]  Donald B. Gennery,et al.  Visual tracking of known three-dimensional objects , 1992, International Journal of Computer Vision.

[32]  Arjan Kuijper,et al.  Robust, fast and accurate vision-based localization of a cooperative target used for space robotic arm , 2017 .

[33]  E. Glenn Lightsey,et al.  An On-Board Image Processing Algorithm for a Spacecraft Optical Navigation Sensor System , 2010 .

[34]  Vincent Lepetit,et al.  Robust 3D Object Tracking from Monocular Images Using Stable Parts , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Guanghui Wang,et al.  Single view based pose estimation from circle or parallel lines , 2008, Pattern Recognit. Lett..

[36]  Roberto Cipolla,et al.  Real-Time Visual Tracking of Complex Structures , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Roberto Opromolla,et al.  Autonomous relative navigation around uncooperative spacecraft based on a single camera , 2019, Aerospace Science and Technology.

[38]  Stefano Soatto,et al.  Structure from Motion Causally Integrated Over Time , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Timothy D. Barfoot,et al.  State Estimation for Robotics , 2017 .

[40]  Abdul Nurunnabi,et al.  Robust statistical approaches for circle fitting in laser scanning three-dimensional point cloud data , 2018, Pattern Recognit..

[41]  Andreas Zell,et al.  An Onboard Monocular Vision System for Autonomous Takeoff, Hovering and Landing of a Micro Aerial Vehicle , 2012, Journal of Intelligent & Robotic Systems.

[42]  Michael E. Taylor,et al.  Differential Geometry I , 1994 .

[43]  Sumant Sharma,et al.  Comparative assessment of techniques for initial pose estimation using monocular vision , 2016 .

[44]  Ahmed M. Elgammal,et al.  Satellite recognition and pose estimation using homeomorphic manifold analysis , 2015, IEEE Transactions on Aerospace and Electronic Systems.

[45]  Eric Marchand,et al.  Vision-based Detection and Tracking for Space Navigation in a Rendezvous Context , 2012, ICRA 2012.

[46]  Roberto Opromolla,et al.  A review of cooperative and uncooperative spacecraft pose determination techniques for close-proximity operations , 2017 .

[47]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[48]  Michael J. Black,et al.  EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation , 1996, International Journal of Computer Vision.

[49]  É. Marchand,et al.  Vision Based Navigation for Debris Removal Missions , 2012 .

[50]  Didier Stricker,et al.  Real-time 3D Camera Tracking for Industrial Augmented Reality Applications , 2005, WSCG.

[51]  Alexander Cropp Pose estimation and relative orbit determination of a nearby target microsatellite using passive imagery , 2001 .

[52]  Bo J. Naasz,et al.  Flight Results from the HST SM4 Relative Navigation Sensor System , 2010 .

[53]  Zhiguo Jiang,et al.  Vision-based pose estimation for cooperative space objects , 2013 .

[54]  Bing Li,et al.  A universal on-orbit servicing system used in the geostationary orbit , 2011 .

[55]  Alberto Del Bimbo,et al.  Metric 3D reconstruction and texture acquisition of surfaces of revolution from a single uncalibrated view , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  S. Seereeram,et al.  Vision-based relative pose estimation for autonomous rendezvous and docking , 2006, 2006 IEEE Aerospace Conference.

[57]  C. Steger OCCLUSION , CLUTTER , AND ILLUMINATION INVARIANT OBJECT RECOGNITION , 2002 .

[58]  Vincent Lepetit,et al.  Stable real-time 3D tracking using online and offline information , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[59]  Pushmeet Kohli,et al.  PoseCut: Simultaneous Segmentation and 3D Pose Estimation of Humans Using Dynamic Graph-Cuts , 2006, ECCV.

[60]  Roberto Opromolla,et al.  Pose Estimation for Spacecraft Relative Navigation Using Model-Based Algorithms , 2017, IEEE Transactions on Aerospace and Electronic Systems.

[61]  Vincent Lepetit,et al.  Learning descriptors for object recognition and 3D pose estimation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[62]  Bei Cheng,et al.  Particle filter-based relative rolling estimation algorithm for non-cooperative infrared spacecraft , 2016 .

[63]  Bin Liang,et al.  A Pose Measurement Method of a Space Noncooperative Target Based on Maximum Outer Contour Recognition , 2020, IEEE Transactions on Aerospace and Electronic Systems.

[64]  David G. Lowe,et al.  Fitting Parameterized Three-Dimensional Models to Images , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[65]  Chang Liu,et al.  Effective ellipse detector with polygonal curve and likelihood ratio test , 2015, IET Comput. Vis..

[66]  Xiangzhi Bai,et al.  Satellite Pose Estimation via Single Perspective Circle and Line , 2018, IEEE Transactions on Aerospace and Electronic Systems.

[67]  Rama Chellappa,et al.  Fast directional chamfer matching , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[68]  Zhengyou Zhang,et al.  Parameter estimation techniques: a tutorial with application to conic fitting , 1997, Image Vis. Comput..

[69]  Éric Marchand,et al.  Real-time markerless tracking for augmented reality: the virtual visual servoing framework , 2006, IEEE Transactions on Visualization and Computer Graphics.

[70]  Coralie D. Jackman,et al.  OPTICAL NAVIGATION CAPABILITIES FOR DEEP SPACE MISSIONS , 2013 .

[71]  Pushmeet Kohli,et al.  Simultaneous Segmentation and Pose Estimation of Humans Using Dynamic Graph Cuts , 2008, International Journal of Computer Vision.

[72]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[73]  W. Marsden I and J , 2012 .

[74]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[75]  Markus Ulrich,et al.  CAD-based recognition of 3D objects in monocular images , 2009, 2009 IEEE International Conference on Robotics and Automation.