A review of cooperative and uncooperative spacecraft pose determination techniques for close-proximity operations

Abstract The capability of an active spacecraft to accurately estimate its relative position and attitude (pose) with respect to an active/inactive, artificial/natural space object (target) orbiting in close-proximity is required to carry out various activities like formation flying, on-orbit servicing, active debris removal, and space exploration. According to the specific mission scenario, the pose determination task involves both theoretical and technological challenges related to the search for the most suitable algorithmic solution and sensor architecture, respectively. As regards the latter aspect, electro-optical sensors represent the best option as their use is compatible with mass and power limitation of micro and small satellites, and their measurements can be processed to estimate all the pose parameters. Overall, the degree of complexity of the challenges related to pose determination largely varies depending on the nature of the targets, which may be actively/passively cooperative, uncooperative but known, or uncooperative and unknown space objects. In this respect, while cooperative pose determination has been successfully demonstrated in orbit, the uncooperative case is still under study by universities, research centers, space agencies and private companies. However, in both the cases, the demand for space applications involving relative navigation maneuvers, also in close-proximity, for which pose determination capabilities are mandatory, is significantly increasing. In this framework, a review of state-of-the-art techniques and algorithms developed in the last decades for cooperative and uncooperative pose determination by processing data provided by electro-optical sensors is herein presented. Specifically, their main advantages and drawbacks in terms of achieved performance, computational complexity, and sensitivity to variability of pose and target geometry, are highlighted.

[1]  David G. Lowe,et al.  Three-Dimensional Object Recognition from Single Two-Dimensional Images , 1987, Artif. Intell..

[2]  Yang Cheng,et al.  Improvement to the Implementation of the QUEST Algorithm , 2014 .

[3]  Phil Palmer,et al.  Pose Estimation of Target Satellite for Proximity Operations , 2000 .

[4]  François Blais Review of 20 years of range sensor development , 2004, J. Electronic Imaging.

[5]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Dariu M Gavrila,et al.  REAL-TIME OBJECT DETECTION USING DISTANCE TRANSFORMS , 1998 .

[7]  Reuben R. Rohrschneider,et al.  Critical Advancement in Telerobotic Servicing Vision Technology , 2010 .

[8]  David W. Miller,et al.  SPHERES flight operations testing and execution , 2009 .

[9]  Bo J. Naasz,et al.  The HST SM4 Relative Navigation Sensor System: Overview and Preliminary Testing Results from the Flight Robotics Lab , 2009 .

[10]  Frank Dellaert,et al.  iSAM: Incremental Smoothing and Mapping , 2008, IEEE Transactions on Robotics.

[11]  Shin-ichiro Nishida,et al.  Space debris removal system using a small satellite , 2006 .

[12]  David K. Geller,et al.  Navigating the Road to Autonomous Orbital Rendezvous , 2007 .

[13]  Francesc Moreno-Noguer,et al.  Very Fast Solution to the PnP Problem with Algebraic Outlier Rejection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Roberto Opromolla,et al.  A Model-Based 3D Template Matching Technique for Pose Acquisition of an Uncooperative Space Object , 2015, Sensors.

[15]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[16]  Klaus Janschek,et al.  EKF-SLAM based Approach for Spacecraft Rendezvous Navigation with Unknown Target Spacecraft , 2010 .

[17]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[18]  Marc Rioux,et al.  Short- and medium-range 3D sensing for space applications , 1997, Defense, Security, and Sensing.

[19]  Chang Liu,et al.  Relative pose estimation for cylinder-shaped spacecrafts using single image , 2014, IEEE Transactions on Aerospace and Electronic Systems.

[20]  Joseph M. Galante,et al.  Pose Measurement Performance of the Argon Relative Navigation Sensor Suite in Simulated Flight Conditions , 2012 .

[21]  S. Seereeram,et al.  Vision-based relative pose estimation for autonomous rendezvous and docking , 2006, 2006 IEEE Aerospace Conference.

[22]  Zhiguo Jiang,et al.  Vision-based pose estimation for cooperative space objects , 2013 .

[23]  Roland Siegwart,et al.  A novel parametrization of the perspective-three-point problem for a direct computation of absolute camera position and orientation , 2011, CVPR 2011.

[24]  Philip David,et al.  SoftPOSIT: Simultaneous Pose and Correspondence Determination , 2002, International Journal of Computer Vision.

[25]  John A. Christian,et al.  Lidar-based relative navigation with respect to non-cooperative objects , 2016 .

[26]  John Leif Jørgensen,et al.  Pose estimation of an uncooperative spacecraft from actual space imagery , 2014 .

[27]  Francois Blais,et al.  Three-dimensional tracking and imaging laser scanner for space operations , 1999, Defense, Security, and Sensing.

[28]  Simone D'Amico,et al.  Prisma Formation Flying Demonstrator: Overview and Conclusions from the Nominal Mission , 2012 .

[29]  V. Lepetit,et al.  EPnP: An Accurate O(n) Solution to the PnP Problem , 2009, International Journal of Computer Vision.

[30]  I. Kawano,et al.  In-orbit demonstration of rendezvous laser radar for unmanned autonomous rendezvous docking , 2004, IEEE Transactions on Aerospace and Electronic Systems.

[31]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[32]  Gerhard Roth,et al.  Video-based system for satellite proximity operations , 2002 .

[33]  Feng Yu,et al.  Stereo-Vision-Based Relative Pose Estimation for the Rendezvous and Docking of Noncooperative Satellites , 2014 .

[34]  Michael Greenspan,et al.  Efficient tracking with the Bounded Hough Transform , 2004, CVPR 2004.

[35]  M. Shuster,et al.  Three-axis attitude determination from vector observations , 1981 .

[36]  Hugh F. Durrant-Whyte,et al.  Simultaneous localization and mapping: part I , 2006, IEEE Robotics & Automation Magazine.

[37]  Éric Marchand,et al.  Real-time markerless tracking for augmented reality: the virtual visual servoing framework , 2006, IEEE Transactions on Visualization and Computer Graphics.

[38]  Jonathan P. How,et al.  Enabling Spacecraft Formation Flying in Any Earth Orbit Through Spaceborne GPS and Enhanced Autonomy Technologies , 2000 .

[39]  Joel A. Hesch,et al.  A Direct Least-Squares (DLS) method for PnP , 2011, 2011 International Conference on Computer Vision.

[40]  Javier Civera,et al.  Inverse Depth Parametrization for Monocular SLAM , 2008, IEEE Transactions on Robotics.

[41]  S. Umeyama,et al.  Least-Squares Estimation of Transformation Parameters Between Two Point Patterns , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Isao Kawano,et al.  In-Orbit Demonstration of an Optical Navigation System for Autonomous Rendezvous Docking , 2011 .

[43]  Giancarmine Fasano,et al.  A Stereo-Vision Based System for Autonomous Navigation of an In-Orbit Servicing Platform , 2009 .

[44]  Uwe D. Hanebeck,et al.  Template matching using fast normalized cross correlation , 2001, SPIE Defense + Commercial Sensing.

[45]  John Leif Jørgensen,et al.  The Advanced Stellar Compass, Development and Operations , 1996 .

[46]  Klaus Schilling,et al.  Relative pose estimation of satellites using PMD-/CCD-sensor data fusion , 2015 .

[47]  Marco Pertile,et al.  Uncertainty evaluation of a vision system for pose measurement of a spacecraft with fiducial markers , 2015, 2015 IEEE Metrology for Aerospace (MetroAeroSpace).

[48]  Avinash C. Kak,et al.  Calculating the 3d-pose of rigid-objects using active appearance models , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[49]  Mohammed Bennamoun,et al.  3D Object Recognition in Cluttered Scenes with Local Surface Features: A Survey , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  P. Gurfil,et al.  Effect of Kinematic Rotation-Translation Coupling on Relative Spacecraft Translational Dynamics , 2009 .

[51]  Takeo Kanade,et al.  A Multiple-Baseline Stereo , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[52]  Jiaolong Yang,et al.  Go-ICP: Solving 3D Registration Efficiently and Globally Optimally , 2013, 2013 IEEE International Conference on Computer Vision.

[53]  Stephane Ruel,et al.  3D LASSO: REAL-TIME POSE ESTIMATION FROM 3D DATA FOR AUTONOMOUS SATELLITE SERVICING , 2005 .

[54]  Tohru Suzuki,et al.  Results of the ETS-7 Mission - Rendezvous Docking and Space Robotics Experiments , 1999 .

[55]  Lance B. Gatrell,et al.  Robust image features: concentric contrasting circles and their image extraction , 1992, Other Conferences.

[56]  Fabien Malbet,et al.  Results of PRISMA/FFIORD extended mission and applicability to future formation flying and active debris removal missions , 2013 .

[57]  F. Markley,et al.  Quaternion Attitude Estimation Using Vector Observations , 2000 .

[58]  R.P. Kornfeld,et al.  New millennium ST6 autonomous rendezvous experiment (ARX) , 2003, 2003 IEEE Aerospace Conference Proceedings (Cat. No.03TH8652).

[59]  Mongi A. Abidi,et al.  A New Efficient and Direct Solution for Pose Estimation Using Quadrangular Targets: Algorithm and Evaluation , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[60]  John Leif Jørgensen,et al.  The Micro Advanced Stellar Compass for ESA's PROBA 2 Mission , 2005 .

[61]  Simone D'Amico,et al.  Autonomous formation flying based on GPS — PRISMA flight results , 2010 .

[62]  R. M. Willett,et al.  Compressed sensing for practical optical imaging systems: A tutorial , 2011, IEEE Photonics Conference 2012.

[63]  Roberto Opromolla,et al.  Performance Evaluation of 3D Model-based Techniques for Autonomous Pose Initialization and Tracking , 2015 .

[64]  Larry S. Davis,et al.  Model-based object pose in 25 lines of code , 1992, International Journal of Computer Vision.

[65]  Ou Ma,et al.  A review of space robotics technologies for on-orbit servicing , 2014 .

[66]  F. Landis Markley,et al.  Attitude Determination Using Two Vector Measurements , 1998 .

[67]  Richard O. Duda,et al.  Use of the Hough transformation to detect lines and curves in pictures , 1972, CACM.

[68]  Gabriella Gaias,et al.  A Far Range Image Processing Method for Autonomous Tracking of an Uncooperative Target , 2013, ICRA 2013.

[69]  Franz Andert,et al.  On the safe navigation problem for unmanned aircraft: Visual odometry and alignment optimizations for UAV positioning , 2014, 2014 International Conference on Unmanned Aircraft Systems (ICUAS).

[70]  D. Lauretta,et al.  OSIRIS-REx asteroid sample-return mission #25 , 2015 .

[71]  Bryan L. Benedict Investing in Satellite Life Extension - Fleet Planning Options for Spacecraft Owner/Operators , 2014 .

[72]  Michael J. Black,et al.  EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation , 1996, International Journal of Computer Vision.

[73]  Michael A. Greenspan,et al.  Model-Based Tracking by Classification in a Tiny Discrete Pose Space , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[74]  Jianliang Tang,et al.  Complete Solution Classification for the Perspective-Three-Point Problem , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[75]  Richard Howard,et al.  Comparison of navigation solutions for autonomous spacecraft from multiple sensor systems , 2008, SPIE Defense + Commercial Sensing.

[76]  Gérard G. Medioni,et al.  Object modelling by registration of multiple range images , 1992, Image Vis. Comput..

[77]  Philip David,et al.  Simultaneous pose and correspondence determination using line features , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[78]  George H. James,et al.  Ladar measurements of the International Space Station , 2001, SPIE Defense + Commercial Sensing.

[79]  Shai Segal,et al.  Stereovision-Based Estimation of Relative Dynamics Between Noncooperative Satellites: Theory and Experiments , 2014, IEEE Transactions on Control Systems Technology.

[80]  Christophe Doignon,et al.  An Introduction to Model-Based Pose Estimation and 3-D Tracking Techniques , 2007 .

[81]  Michele Grassi,et al.  Relative Navigation in LEO by Carrier-Phase Differential GPS with Intersatellite Ranging Augmentation , 2013 .

[82]  E. Kley,et al.  Efficient extreme ultraviolet transmission gratings for plasma diagnostics , 2011 .

[83]  K. S. Arun,et al.  Least-Squares Fitting of Two 3-D Point Sets , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[84]  Heike Benninghoff,et al.  Rendezvous Involving a Non-Cooperative, Tumbling Target - Estimation of Moments of Inertia and Center of Mass of an Unknown Target , 2015 .

[85]  Richard T. Howard,et al.  Smartphone Video Guidance Sensor for Small Satellites , 2013 .

[86]  É. Marchand,et al.  Vision Based Navigation for Debris Removal Missions , 2012 .

[87]  John Leif Jørgensen,et al.  Short Range Pose and Position Determination of Spacecraft Using a μ-Advanced Stellar Compass , 2008 .

[88]  Francois Blais,et al.  Eye-safe digital 3-D sensing for space applications , 2000 .

[89]  Long Quan,et al.  Linear N-Point Camera Pose Determination , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[90]  Frank Crosby,et al.  Multiphase Segmentation of 3D Flash Lidar Images , 2011 .

[91]  Steven Dubowsky,et al.  State, shape, and parameter estimation of space objects from range images , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[92]  Scott P. Cryan,et al.  A Survey of LIDAR Technology and Its Use in Spacecraft Relative Navigation , 2013 .

[93]  Steven Gold,et al.  A Graduated Assignment Algorithm for Graph Matching , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[94]  Andreas Fleischner,et al.  Pose Tracking of a Noncooperative Spacecraft During Docking Maneuvers Using a Time-of-Flight Sensor , 2016 .

[95]  Itzhack Y. Bar-Itzhack,et al.  Optimized TRIAD Algorithm for Attitude Determination , 1996 .

[96]  Luke Walker,et al.  Automated Proximity Operations Using Image-Based Relative Navigation , 2012 .

[97]  Richard Howard,et al.  Effects of optical artifacts in a laser-based spacecraft navigation sensor , 2007, SPIE Defense + Commercial Sensing.

[98]  Michèle Lavagna,et al.  Stereovision-based pose and inertia estimation of unknown and uncooperative space objects , 2017 .

[99]  John J. Leonard,et al.  Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age , 2016, IEEE Transactions on Robotics.

[100]  Robert M. Haralick,et al.  Statistical estimation for exterior orientation from line-to-line correspondences , 1996, Image Vis. Comput..

[101]  Robert B. Friend,et al.  Orbital Express program summary and mission overview , 2008, SPIE Defense + Commercial Sensing.

[102]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[103]  Christopher D'Souza,et al.  Cooperative Relative Navigation of Spacecraft Using Flash Light Detection and Ranging Sensors , 2014 .

[104]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using orthonormal matrices , 1988 .

[105]  Roberto Opromolla,et al.  Pose Estimation for Spacecraft Relative Navigation Using Model-Based Algorithms , 2017, IEEE Transactions on Aerospace and Electronic Systems.

[106]  Horst Bischof,et al.  Pose Estimation of Known Objects by Efficient Silhouette Matching , 2010, 2010 20th International Conference on Pattern Recognition.

[107]  Stephen M. Rock,et al.  Improved frame-to-frame pose tracking during vision-only SLAM/SFM with a tumbling target , 2011, 2011 IEEE International Conference on Robotics and Automation.

[108]  Richard T. Howard,et al.  Orbital Express AVGS Validation and Calibration for Automated Rendezvous , 2008 .

[109]  Danny Gibbins,et al.  Multi-scale Feature Extraction for 3D Models Using Local Surface Curvature , 2008, 2008 Digital Image Computing: Techniques and Applications.

[110]  Wigbert Fehse Automated Rendezvous and Docking of Spacecraft: The drivers for the approach strategy , 2003 .

[111]  J. Kawaguchi,et al.  Robotics and autonomous technology for asteroid sample return mission , 2005, ICAR '05. Proceedings., 12th International Conference on Advanced Robotics, 2005..

[112]  Alvar Saenz-Otero,et al.  Engineering Notes Relative Computer Vision-Based Navigation for Small Inspection Spacecraft , 2015 .

[113]  Bin Zhang,et al.  A TSR Visual Servoing System Based on a Novel Dynamic Template Matching Method † , 2015, Sensors.

[114]  Zhanyi Hu,et al.  PnP Problem Revisited , 2005, Journal of Mathematical Imaging and Vision.

[115]  Alexander Cropp,et al.  PROBA-3: Precise formation flying demonstration mission , 2013 .

[116]  Nourain Dawoud Nadir,et al.  Fast Template Matching Method Based Optimized Sum of Absolute Difference Algorithm for Face Localization , 2011 .

[117]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[118]  Brent E. Tweddle,et al.  Computer vision-based localization and mapping of an unknown, uncooperative and spinning target for spacecraft proximity operations , 2013 .

[119]  John A. Christian,et al.  Comparison of Orion Vision Navigation Sensor Performance from STS-134 and the Space Operations Simulation Center , 2012 .

[120]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[121]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[122]  Eberhard Gill,et al.  PRISMA - AN AUTONOMOUS FORMATION FLYING MISSION , 2006 .

[123]  Anton van den Hengel,et al.  Thrift: Local 3D Structure Recognition , 2007, 9th Biennial Conference of the Australian Pattern Recognition Society on Digital Image Computing Techniques and Applications (DICTA 2007).

[124]  Stephane Ruel,et al.  TriDAR: A HYBRID SENSOR FOR EXPLOITING THE COMPLEMENTARY NATURE OF TRIANGULATION AND LIDAR TECHNOLOGIES , 2005 .

[125]  Mario Vento,et al.  Thirty Years Of Graph Matching In Pattern Recognition , 2004, Int. J. Pattern Recognit. Artif. Intell..

[126]  Yuming Bo,et al.  Point Cloud Based Relative Pose Estimation of a Satellite in Close Range , 2016, Sensors.

[127]  Klaus Janschek,et al.  Experimental results for image-based geometrical reconstruction for spacecraft Rendezvous navigation with unknown and uncooperative target spacecraft , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[128]  Giovanni B. Palmerini,et al.  Ground tests for vision based determination and control of formation flying spacecraft trajectories , 2014 .

[129]  Christophe Bonnal,et al.  Active debris removal: Recent progress and current trends , 2013 .

[130]  Wenfu Xu,et al.  Pose measurement of large non-cooperative satellite based on collaborative cameras , 2011 .

[131]  Richard T. Howard,et al.  The Advanced Video Guidance Sensor: Orbital Express and the Next Generation , 2008 .

[132]  Sumant Sharma,et al.  Comparative assessment of techniques for initial pose estimation using monocular vision , 2016 .

[133]  Richard T. Howard,et al.  Next Generation Advanced Video Guidance Sensor Development and Test , 2009 .

[134]  Roberto Cipolla,et al.  Real-Time Visual Tracking of Complex Structures , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[135]  Xinhua Zhuang,et al.  Pose estimation from corresponding point data , 1989, IEEE Trans. Syst. Man Cybern..

[136]  Hamid Hemmati,et al.  Laser radar for spacecraft guidance applications , 2003, 2003 IEEE Aerospace Conference Proceedings (Cat. No.03TH8652).

[137]  John A. Christian,et al.  Pattern Design for 3D Point Matching , 2015 .

[138]  Roberto Opromolla,et al.  Uncooperative pose estimation with a LIDAR-based system , 2015 .

[139]  David K. Geller,et al.  Relative Angles-Only Navigation and Pose Estimation for Autonomous Orbital Rendezvous , 2006 .

[140]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[141]  Gregory W. Heckler,et al.  GPS Navigation for the Magnetospheric Multi-Scale Mission , 2009 .

[142]  Mohammed Bennamoun,et al.  A Comprehensive Performance Evaluation of 3D Local Feature Descriptors , 2015, International Journal of Computer Vision.

[143]  Peter Cheeseman,et al.  On the Representation and Estimation of Spatial Uncertainty , 1986 .

[144]  Tong Pan,et al.  Autonomous satellite rendezvous and docking using lidar and model based vision , 2005, SPIE Defense + Commercial Sensing.

[145]  Timothy E. Rumford Demonstration of autonomous rendezvous technology (DART) project summary , 2003, SPIE Defense + Commercial Sensing.

[146]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[147]  David Fofi,et al.  A review of recent range image registration methods with accuracy evaluation , 2007, Image Vis. Comput..

[148]  Vivek K Goyal,et al.  Exploiting sparsity in time-of-flight range acquisition using a single time-resolved sensor. , 2011, Optics express.

[149]  J. G. Semple,et al.  Algebraic Projective Geometry , 1953 .

[150]  Francois Blais,et al.  Neptec 3D Laser Camera System: from space mission STS-105 to terrestrial applications , 2002 .

[151]  Eric Marchand,et al.  Vision-based Detection and Tracking for Space Navigation in a Rendezvous Context , 2012, ICRA 2012.

[152]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[153]  John A. Christian,et al.  LIDAR-based Relative Navigation of Non-Cooperative Objects Using Point Cloud Descriptors , 2016 .

[154]  Steven Dubowsky,et al.  Shape, Motion, and Parameter Estimation of Flexible Space Structures using Laser Rangefinders , 2005, Robotics: Science and Systems.

[155]  Eric Mjolsness,et al.  New Algorithms for 2D and 3D Point Matching: Pose Estimation and Correspondence , 1998, NIPS.