Tigris: Architecture and Algorithms for 3D Perception in Point Clouds

Machine perception applications are increasingly moving toward manipulating and processing 3D point cloud. This paper focuses on point cloud registration, a key primitive of 3D data processing widely used in high-level tasks such as odometry, simultaneous localization and mapping, and 3D reconstruction. As these applications are routinely deployed in energy-constrained environments, real-time and energy-efficient point cloud registration is critical. We present Tigris, an algorithm-architecture co-designed system specialized for point cloud registration. Through an extensive exploration of the registration pipeline design space, we find that, while different design points make vastly different trade-offs between accuracy and performance, KD-tree search is a common performance bottleneck, and thus is an ideal candidate for architectural specialization. While KD-tree search is inherently sequential, we propose an acceleration-amenable data structure and search algorithm that exposes different forms of parallelism of KD-tree search in the context of point cloud registration. The co-designed accelerator systematically exploits the parallelism while incorporating a set of architectural techniques that further improve the accelerator efficiency. Overall, Tigris achieves 77.2× speedup and 7.4× power reduction in KD-tree search over an RTX 2080 Ti GPU, which translates to a 41.7% registration performance improvements and 3.0× power reduction.

[1]  Qiang Wu,et al.  A Coarse-to-Fine Algorithm for Matching and Registration in 3D Cross-Source Point Clouds , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Martin D. Levine,et al.  Registering Multiview Range Data to Create 3D Computer Objects , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  David Zhang,et al.  Three-dimensional surface registration: A neural network strategy , 2006, Neurocomputing.

[4]  Gene H. Golub,et al.  Singular value decomposition and least squares solutions , 1970, Milestones in Matrix Computation.

[5]  Ji Zhang,et al.  Visual-lidar odometry and mapping: low-drift, robust, and fast , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[6]  Shihua Li,et al.  Tree point clouds registration using an improved ICP algorithm based on kd-tree , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[7]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[8]  Federico Tombari,et al.  Unique Signatures of Histograms for Local Surface Description , 2010, ECCV.

[9]  Zidong Wang,et al.  On global exponential stability of generalized stochastic neural networks with mixed time-delays , 2006, Neurocomputing.

[10]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[11]  Jean-Emmanuel Deschaud,et al.  IMLS-SLAM: Scan-to-Model Matching Based on 3D Data , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[12]  Brent Schwarz,et al.  LIDAR: Mapping the world in 3D , 2010 .

[13]  Meng Liu,et al.  Efficient Mean‐shift Clustering Using Gaussian KD‐Tree , 2010, Comput. Graph. Forum.

[14]  Mubarak Shah,et al.  A 3-dimensional sift descriptor and its application to action recognition , 2007, ACM Multimedia.

[15]  H. T. Kung Why systolic architectures? , 1982, Computer.

[16]  Markus H. Gross,et al.  A hardware processing unit for point sets , 2008, GH '08.

[17]  Hideharu Amano,et al.  An FPGA Acceleration for the Kd-tree Search in Photon Mapping , 2013, ARC.

[18]  Silvio Savarese,et al.  DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Michael A. Greenspan,et al.  Approximate k-d tree search for efficient ICP , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[20]  Rahul Khanna,et al.  RAPL: Memory power estimation and capping , 2010, 2010 ACM/IEEE International Symposium on Low-Power Electronics and Design (ISLPED).

[21]  Vladlen Koltun,et al.  Robust reconstruction of indoor scenes , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Cyrill Stachniss,et al.  Efficient Surfel-Based SLAM using 3D Laser Range Data in Urban Environments , 2018, Robotics: Science and Systems.

[23]  Jon Louis Bentley,et al.  An Algorithm for Finding Best Matches in Logarithmic Expected Time , 1977, TOMS.

[24]  Vladlen Koltun,et al.  Colored Point Cloud Registration Revisited , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[25]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[26]  Pat Hanrahan,et al.  Photon mapping on programmable graphics hardware , 2003, HWWS '03.

[27]  Jonathan Dyssel Stets,et al.  Visualization and labeling of point clouds in virtual reality , 2017, SIGGRAPH ASIA.

[28]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[29]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[30]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[31]  Shi-Min Hu,et al.  Efficient affinity-based edit propagation using K-D tree , 2009, ACM Trans. Graph..

[32]  Benjamin Bustos,et al.  Harris 3D: a robust extension of the Harris operator for interest point detection on 3D meshes , 2011, The Visual Computer.

[33]  Jitendra Malik,et al.  Recognizing Objects in Range Data Using Regional Point Descriptors , 2004, ECCV.

[34]  Zhiguo Cao,et al.  A fast and robust local descriptor for 3D point cloud registration , 2016, Inf. Sci..

[35]  George Vosselman,et al.  3D BUILDING MODEL RECONSTRUCTION FROM POINT CLOUDS AND GROUND PLANS , 2001 .

[36]  Michael D. McCool,et al.  Low latency photon mapping using block hashing , 2002, HWWS '02.

[37]  Michal Irani,et al.  Recovery of Ego-Motion Using Region Alignment , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Claus Brenner,et al.  3D feature point extraction from LiDAR data using a neural network , 2016 .

[39]  George A. Constantinides,et al.  FPGA-based K-means clustering using tree-based data structures , 2013, 2013 23rd International Conference on Field programmable Logic and Applications.

[40]  Sunil Arya,et al.  An optimal algorithm for approximate nearest neighbor searching fixed dimensions , 1998, JACM.

[41]  Paolo Cignoni,et al.  A low cost 3D scanner based on structured light , 2001 .

[42]  Hans-Peter Seidel,et al.  Stackless KD‐Tree Traversal for High Performance GPU Ray Tracing , 2007, Comput. Graph. Forum.

[43]  Jun Cheng,et al.  Registration for 3-D point cloud using angular-invariant feature , 2009, Neurocomputing.

[44]  David G. Lowe,et al.  Scalable Nearest Neighbor Algorithms for High Dimensional Data , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Mark Whitty,et al.  Autonomous Navigation using a Real-Time 3 D Point Cloud , 2010 .

[46]  Laurent Miclet,et al.  Approximative fast nearest-neighbour recognition , 1983, Pattern Recognit. Lett..

[47]  S. Burak Gokturk,et al.  A Time-Of-Flight Depth Sensor - System Description, Issues and Solutions , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[48]  Pietro Perona,et al.  Distributed Kd-Trees for Retrieval from Very Large Image Collections , 2011 .

[49]  Xu Cheng,et al.  An automatic and robust point cloud registration framework based on view-invariant local feature descriptors and transformation consistency verification , 2017 .

[50]  Andreas Nüchter,et al.  GPU-Accelerated Nearest Neighbor Search for 3D Registration , 2009, ICVS.

[51]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[52]  Fabian Gieseke,et al.  Buffer k-d Trees: Processing Massive Nearest Neighbor Queries on GPUs , 2014, ICML.

[53]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[54]  Sing Bing Kang,et al.  Registration and integration of textured 3-D data , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[55]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[56]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[57]  Beng Chin Ooi,et al.  Spatial kd-Tree: A Data Structure for Geographic Database , 1987, BTW.

[58]  Gérard G. Medioni,et al.  Object modelling by registration of multiple range images , 1992, Image Vis. Comput..

[59]  Sridha Sridharan,et al.  Probabilistic Surfel Fusion for Dense LiDAR Mapping , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[60]  C. Brenner,et al.  3 D FEATURE POINT EXTRACTION FROM LIDAR DATA USING A NEURAL NETWORK , 2016 .

[61]  Zoltan-Csaba Marton,et al.  Tutorial: Point Cloud Library: Three-Dimensional Object Recognition and 6 DOF Pose Estimation , 2012, IEEE Robotics & Automation Magazine.

[62]  Anath Fischer,et al.  3D Point Cloud Registration for Localization Using a Deep Neural Network Auto-Encoder , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[63]  Paul J. Besl,et al.  Method for registration of 3-D shapes , 1992, Other Conferences.

[64]  Liang-Gee Chen,et al.  Video stabilization for vehicular applications using SURF-like descriptor and KD-tree , 2010, 2010 IEEE International Conference on Image Processing.

[65]  Pat Hanrahan,et al.  Interactive k-d tree GPU raytracing , 2007, SI3D.

[66]  Ji Zhang,et al.  LOAM: Lidar Odometry and Mapping in Real-time , 2014, Robotics: Science and Systems.

[67]  J. J. Moré,et al.  Levenberg--Marquardt algorithm: implementation and theory , 1977 .

[68]  Martin Buss,et al.  Comparison of surface normal estimation methods for range sensing applications , 2009, 2009 IEEE International Conference on Robotics and Automation.

[69]  Abhinav Gupta,et al.  Designing deep networks for surface normal estimation , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[70]  R. Weisberg A-N-D , 2011 .

[71]  Alexandre Boulch,et al.  Deep Learning for Robust Normal Estimation in Unstructured Point Clouds , 2016, Comput. Graph. Forum.

[72]  Wolfram Burgard,et al.  Point feature extraction on 3D range scans taking into account object boundaries , 2011, 2011 IEEE International Conference on Robotics and Automation.

[73]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[74]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.