Deep Learning for Robust Normal Estimation in Unstructured Point Clouds

Normal estimation in point clouds is a crucial first step for numerous algorithms, from surface reconstruction and scene understanding to rendering. A recurrent issue when estimating normals is to make appropriate decisions close to sharp features, not to smooth edges, or when the sampling density is not uniform, to prevent bias. Rather than resorting to manually‐designed geometric priors, we propose to learn how to make these decisions, using ground‐truth data made from synthetic scenes. For this, we project a discretized Hough space representing normal directions onto a structure amenable to deep learning. The resulting normal estimation method outperforms most of the time the state of the art regarding robustness to outliers, to noise and to point density variation, in the presence of sharp edges, while remaining fast, scaling up to millions of points.

[1]  K. Lingemann,et al.  The 3D Hough Transform for plane detection in point clouds: A review and a new accumulator design , 2011 .

[2]  Alexandre Boulch,et al.  Fast and Robust Normal Estimation for Point Clouds with Sharp Features , 2012, Comput. Graph. Forum.

[3]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[4]  Nancy Argüelles,et al.  Author ' s , 2008 .

[5]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  François Fleuret,et al.  Improving Object Classification using Pose Information , 2012 .

[7]  Tony DeRose,et al.  Surface reconstruction from unorganized points , 1992, SIGGRAPH.

[8]  Cohen-OrDaniel,et al.  ℓ1-Sparse reconstruction of sharp point set surfaces , 2010 .

[9]  Fei Shen,et al.  Corner detection based on modified Hough transform , 2002, Pattern Recognit. Lett..

[10]  Matthias Zwicker,et al.  Surface splatting , 2001, SIGGRAPH.

[11]  Markus H. Gross,et al.  Feature Preserving Point Set Surfaces based on Non‐Linear Kernel Regression , 2009, Comput. Graph. Forum.

[12]  HoppeHugues,et al.  Surface Reconstruction from Unorganized Points , 1992 .

[13]  Marc Alexa,et al.  Computing and Rendering Point Set Surfaces , 2003, IEEE Trans. Vis. Comput. Graph..

[14]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[16]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[17]  Ben Graham,et al.  Sparse 3D convolutional neural networks , 2015, BMVC.

[18]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[19]  Guannan Gao,et al.  Probabilistic Hough Transform , 2011 .

[20]  Jian Liu,et al.  Point cloud normal estimation via low-rank subspace clustering , 2013, Comput. Graph..

[21]  Bao Li,et al.  Robust normal estimation for point clouds with sharp features , 2010, Comput. Graph..

[22]  Song Wu,et al.  3 D ShapeNets : A Deep Representation for Volumetric Shape Modeling , 2015 .

[23]  Hans-Peter Seidel,et al.  MovieReshape: tracking and reshaping of humans in videos , 2010, SIGGRAPH 2010.

[24]  Erkki Oja,et al.  Randomized hough transform (rht) : Basic mech-anisms, algorithms, and computational complexities , 1993 .

[25]  Luc Van Gool,et al.  Hough Transform and 3D SURF for Robust Three Dimensional Classification , 2010, ECCV.

[26]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Dana H. Ballard,et al.  Generalizing the Hough transform to detect arbitrary shapes , 1981, Pattern Recognit..

[28]  Daniel Cohen-Or,et al.  Consolidation of unorganized point clouds for surface reconstruction , 2009, ACM Trans. Graph..

[29]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Tamal K. Dey,et al.  Provable surface reconstruction from noisy samples , 2004, SCG '04.

[31]  Abhinav Gupta,et al.  Designing deep networks for surface normal estimation , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Hans-Peter Seidel,et al.  Neural meshes: statistical learning based on normals , 2003, 11th Pacific Conference onComputer Graphics and Applications, 2003. Proceedings..

[33]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[34]  Niloy J. Mitra,et al.  Estimating surface normals in noisy point cloud data , 2003, SCG '03.

[35]  Pierre Alliez,et al.  Eurographics Symposium on Geometry Processing (2007) Voronoi-based Variational Reconstruction of Unoriented Point Sets , 2022 .

[36]  Markus H. Gross,et al.  Algebraic point set surfaces , 2007, ACM Trans. Graph..

[37]  Leonidas J. Guibas,et al.  Estimating surface normals in noisy point cloud data , 2004, Int. J. Comput. Geom. Appl..

[38]  Alexandre Boulch,et al.  Piecewise‐Planar 3D Reconstruction with Edge and Corner Regularization , 2014, Comput. Graph. Forum.

[39]  Saburo Tsuji,et al.  Detection of Ellipses by a Modified Hough Transformation , 1978, IEEE Transactions on Computers.

[40]  Jie Zhang,et al.  Quality point cloud normal estimation by guided least squares representation , 2015, Comput. Graph..

[41]  Bidyut Baran Chaudhuri,et al.  A survey of Hough Transform , 2015, Pattern Recognit..

[42]  Chunhua Shen,et al.  Depth and surface normal estimation from monocular images using regression on deep features and hierarchical CRFs , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  F. Pirri,et al.  Point Cloud Segmentation and 3 D Path Planning for Tracked Vehicles in Cluttered and Dynamic Environments , 2014 .

[44]  Marc Levoy,et al.  QSplat: a multiresolution point rendering system for large meshes , 2000, SIGGRAPH.

[45]  Björn Stenger,et al.  A new distance for scale-invariant 3D shape recognition and registration , 2011, 2011 International Conference on Computer Vision.

[46]  R. E. Application of the generalised Hough transform to corner detection-Computers and Digital Techniques [see also IEE Proceedings-Computers and Digital Techniques], IEE , 2004 .

[47]  Markus H. Gross,et al.  Shape modeling with point-sampled geometry , 2003, ACM Trans. Graph..

[48]  Jean-Philippe Pons,et al.  Robust piecewise-planar 3D reconstruction and completion from large-scale unstructured point data , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[49]  E. R. Davies Application of the generalised Hough transform to corner detection , 1988 .

[50]  Richard O. Duda,et al.  Use of the Hough transformation to detect lines and curves in pictures , 1972, CACM.

[51]  Daniel Cohen-Or,et al.  ℓ1-Sparse reconstruction of sharp point set surfaces , 2010, TOGS.

[52]  Wenping Wang,et al.  Denoising point sets via L0 minimization , 2015, Comput. Aided Geom. Des..

[53]  Marc Pouget,et al.  Estimating differential quantities using polynomial fitting of osculating jets , 2003, Comput. Aided Geom. Des..

[54]  Abhinav Gupta,et al.  Marr Revisited: 2D-3D Alignment via Surface Normal Prediction , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).