Satellite Pose Estimation Challenge: Dataset, Competition Design, and Results

Reliable pose estimation of uncooperative satellites is a key technology for enabling future on-orbit servicing and debris removal missions. The Kelvins Satellite Pose Estimation Challenge aims at evaluating and comparing monocular-vision-based approaches and pushing the state of the art on this problem. This article is based on the Satellite Pose Estimation Dataset, the first publicly available machine learning set of synthetic and real spacecraft imageries. The choice of dataset reflects one of the unique challenges associated with spaceborne computer vision tasks, namely the lack of spaceborne images to train and validate the developed algorithms. This article briefly reviews the basic properties and the collection process of the dataset that was made publicly available. The competition design, including the definition of performance metrics and the adopted testbed, is also discussed. The main contribution of this article is the analysis of the submissions of the 48 competitors, which compares the performance of different approaches and uncovers what factors make the satellite pose estimation problem especially challenging.

[1]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[2]  Dana H. Ballard,et al.  Generalizing the Hough transform to detect arbitrary shapes , 1981, Pattern Recognit..

[3]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[5]  Alexander Cropp.  Pose estimation and relative orbit determination of a nearby target microsatellite using passive imagery , 2001 .

[6]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[7]  Xibin Cao,et al.  Closed‐form solution of monocular vision‐based relative pose determination for RVD spacecrafts , 2005 .

[8]  Son-Goo Kim,et al.  Kalman filtering for relative spacecraft attitude and position estimation , 2005 .

[9]  V. Lepetit,et al.  EPnP: An Accurate O(n) Solution to the PnP Problem , 2009, International Journal of Computer Vision.

[10]  Manny R. Leinz,et al.  Orbital Express Autonomous Rendezvous and Capture Sensor System (ARCSS) flight test results , 2008, SPIE Defense + Commercial Sensing.

[11]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[12]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[13]  Jia Deng,et al.  A large-scale hierarchical image database , 2009, CVPR 2009.

[14]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[15]  Éric Marchand,et al.  Vision-based space autonomous rendezvous: A case study , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Vincent Lepetit,et al.  Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes , 2012, ACCV.

[17]  É. Marchand,et al.  Vision Based Navigation for Debris Removal Missions , 2012 .

[18]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[19]  David Barnhart,et al.  DARPA Phoenix Payload Orbital Delivery System (PODs): “FedEx to GEO” , 2013 .

[20]  John Leif Jørgensen,et al.  Pose estimation of an uncooperative spacecraft from actual space imagery , 2014 .

[21]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[22]  Roberto Cipolla,et al.  Convolutional networks for real-time 6-DOF camera relocalization , 2015, ArXiv.

[23]  Leonidas J. Guibas,et al.  Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[24]  Jitendra Malik,et al.  Viewpoints and keypoints , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Roberto Cipolla,et al.  PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[26]  Alessio Grompone,et al.  Vision-Based 3D Motion Estimation for On-Orbit Proximity Satellite Tracking and Navigation , 2015 .

[27]  Eric Joffre,et al.  RemoveDEBRIS: An in-orbit active debris removal demonstration mission , 2016 .

[28]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Eric Brachmann,et al.  Uncertainty-Driven 6D Pose Estimation of Objects and Scenes from a Single RGB Image , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Benjamin B. Reed,et al.  The Restore-L Servicing Mission , 2016 .

[32]  Nassir Navab,et al.  SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[33]  Sumant Sharma,et al.  REDUCED-DYNAMICS POSE ESTIMATION FOR NON-COOPERATIVE SPACECRAFT RENDEZVOUS USING MONOCULAR VISION , 2017 .

[34]  René Vidal,et al.  3D Pose Regression Using Convolutional Neural Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[35]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Vincent Lepetit,et al.  BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[37]  Cewu Lu,et al.  Estimating 6D Pose From Localizing Designated Surface Keypoints , 2018, ArXiv.

[38]  K. Durth,et al.  Prisma , 2018, Homiletische Monatshefte.

[39]  Dieter Fox,et al.  PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes , 2017, Robotics: Science and Systems.

[40]  Simone D'Amico,et al.  Pose estimation for non-cooperative spacecraft rendezvous using convolutional neural networks , 2018, 2018 IEEE Aerospace Conference.

[41]  Zoltan-Csaba Marton,et al.  Implicit 3D Orientation Learning for 6D Object Detection from RGB Images , 2018, ECCV.

[42]  Pascal Fua,et al.  Real-Time Seamless Single Shot 6D Object Pose Prediction , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[43]  Simone D'Amico,et al.  Robust Model-Based Monocular Pose Initialization for Noncooperative Spacecraft Rendezvous , 2018, Journal of Spacecraft and Rockets.

[44]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[45]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[46]  Hujun Bao,et al.  PVNet: Pixel-Wise Voting Network for 6DoF Pose Estimation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Tat-Jun Chin,et al.  Satellite Pose Estimation with Deep Landmark Regression and Nonlinear Pose Refinement , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[48]  Simone D'Amico,et al.  Pose Estimation for Non-Cooperative Rendezvous Using Neural Networks , 2019, ArXiv.

[49]  Dong Liu,et al.  High-Resolution Representations for Labeling Pixels and Regions , 2019, ArXiv.

[50]  Simone D'Amico,et al.  Towards Robust Learning-Based Pose Estimation of Noncooperative Spacecraft , 2019, ArXiv.

[51]  Simone D'Amico,et al.  Variable-Magnification Optical Stimulator for Training and Validation of Spaceborne Vision-Based Navigation , 2019, Journal of Spacecraft and Rockets.

[52]  Pascal Fua,et al.  Segmentation-Driven 6D Object Pose Estimation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Eberhard Gill,et al.  Review of the robustness and applicability of monocular pose estimation systems for relative navigation with an uncooperative spacecraft , 2019, Progress in Aerospace Sciences.

[54]  Sumant Sharma,et al.  POSE ESTIMATION FOR NON-COOPERATIVE SPACECRAFT RENDEZVOUS USING NEURAL NETWORKS , 2019 .

[55]  Yang Gao,et al.  Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering , 2019, 2020 IEEE International Conference on Robotics and Automation (ICRA).