HW/SW Codesign and FPGA Acceleration of Visual Odometry Algorithms for Rover Navigation on Mars

Future Mars exploration missions rely heavily on high-mobility autonomous rovers equipped with sophisticated scientific instruments and possessing advanced navigational capabilities. Increasing their navigation velocity and localization accuracy is essential for enabling these rovers to explore large areas on Mars. Contemporary Mars rovers move slowly, partially due to the long execution time of complex computer vision algorithms running on their slow space-grade CPUs. This paper exploits the advent of high-performance space-grade field-programmable gate arrays (FPGAs) to accelerate the navigation of future rovers. Specifically, it focuses on visual odometry (VO) and performs HW/SW codesign to achieve one order of magnitude faster execution and improved accuracy. Conforming to the specifications of the European Space Agency, we build a proof-of-concept system on an HW/SW platform with processing power resembling that to be available onboard future rovers. We develop a codesign methodology adapted to the rover's specifications, design parallel architectures, and customize several feature extraction, matching, and motion estimation algorithms. We implement and evaluate five distinct HW/SW pipelines on a Virtex6 FPGA and a 150 MIPS CPU. We provide a detailed analysis of their cost-time-accuracy tradeoffs and quantify the benefits of employing FPGAs for implementing VO. Our solution achieves a speedup factor of 16× over a CPU-only implementation, handling a stereo image pair in less than 1 s, with a 1.25% mean positional error after a 100 m traverse and an FPGA cost of 54 K LUTs and 1.46-MB RAM.

[1]  Dah-Jye Lee,et al.  The Nature-Inspired BASIS Feature Descriptor for UAV Imagery and Its Hardware Implementation , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Niccolò Battezzati,et al.  SURF algorithm in FPGA: A novel architecture for high demanding industrial applications , 2012, 2012 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[3]  Mark Woods,et al.  Seeker—Autonomous Long‐range Rover Navigation for Remote Exploration , 2014, J. Field Robotics.

[4]  Heiko Hirschmüller,et al.  Stereo vision and IMU based real-time ego-motion and depth image computation on a handheld device , 2013, 2013 IEEE International Conference on Robotics and Automation.

[5]  Zhiguo Jiang,et al.  An efficient hardware architecture of the optimised SIFT descriptor generation , 2012, 22nd International Conference on Field Programmable Logic and Applications (FPL).

[6]  Teodor Tomic,et al.  Autonomous Vision‐based Micro Air Vehicle for Indoor and Outdoor Navigation , 2014, J. Field Robotics.

[7]  Andrew E. Johnson,et al.  Computer Vision on Mars , 2007, International Journal of Computer Vision.

[8]  Y. Kuwata,et al.  Enabling continuous planetary rover navigation through FPGA stereo and visual odometry , 2012, 2012 IEEE Aerospace Conference.

[9]  Tim D. Barfoot,et al.  Online visual motion estimation using FastSLAM with SIFT features , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Guangjun Zhang,et al.  SIFT Hardware Implementation for Real-Time Image Feature Extraction , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[12]  Friedrich Fraundorfer,et al.  Visual Odometry Part I: The First 30 Years and Fundamentals , 2022 .

[13]  Andrew Zisserman,et al.  Feature Based Methods for Structure and Motion Estimation , 1999, Workshop on Vision Algorithms.

[14]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[15]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[16]  Dimitrios Soudris,et al.  SPARTAN: Developing a Vision System for Future Autonomous Space Exploration Robots , 2014, J. Field Robotics.

[17]  Zhiguo Cao,et al.  Horizontal velocity estimation via downward looking descent images for lunar landing , 2014, IEEE Transactions on Aerospace and Electronic Systems.

[18]  Alexandru Tupan,et al.  Triangulation , 1997, Comput. Vis. Image Underst..

[19]  Vincent Lepetit,et al.  BRIEF: Binary Robust Independent Elementary Features , 2010, ECCV.

[20]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[21]  Gregory D. Hager,et al.  Fast and Globally Convergent Pose Estimation from Video Images , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Zhiguo Cao,et al.  An Embedded System-on-Chip Architecture for Real-time Visual Detection and Matching , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[24]  Vincent Lepetit,et al.  BRIEF: Computing a Local Binary Descriptor Very Fast , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Joachim M. Buhmann,et al.  Empirical Evaluation of Dissimilarity Measures for Color and Texture , 2001, Comput. Vis. Image Underst..

[26]  W. James MacLean,et al.  Field trial results of planetary rover visual motion estimation in Mars analogue terrain , 2012, J. Field Robotics.

[27]  Libor Preucil,et al.  FPGA-based module for SURF extraction , 2014, Machine Vision and Applications.

[28]  Yung-Chang Chen,et al.  High-Performance SIFT Hardware Accelerator for Real-Time Image Feature Extraction , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[29]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[30]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[31]  John Enright,et al.  The Devon Island rover navigation dataset , 2012, Int. J. Robotics Res..

[32]  Ioannis Papaefstathiou,et al.  Fast and Efficient FPGA-Based Feature Detection Employing the SURF Algorithm , 2010, 2010 18th IEEE Annual International Symposium on Field-Programmable Custom Computing Machines.

[33]  Larry H. Matthies,et al.  Robust and Efficient Stereo Feature Tracking for Visual Odometry , 2008, 2008 IEEE International Conference on Robotics and Automation.

[34]  Tom Drummond,et al.  Faster and Better: A Machine Learning Approach to Corner Detection , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Maria E. Angelopoulou,et al.  Vision-Based Egomotion Estimation on FPGA for Unmanned Aerial Vehicle Navigation , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[36]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .