Statistical approaches to multi-scale point cloud processing

In recent years, 3D geometry has gained increasing popularity as the new form of digital media content. Due to advances in sensor technology, it is now feasible to acquire highly detailed 3D scans of complex scenes to obtain millions of data points at high sampling rates over large spatial extents. This ability to acquire high-resolution depth information brings with it the possibility of using 3D geometric data to construct detailed shape models and of perhaps combining 3D depth with visual appearance from images to address challenging problems in computer vision. However, geometric information represented as a 3D point cloud presents challenges uniquely different from other data modalities such as images or audio. Due to a combination of reasons such as the spatial irregularity of the data and the implicit nature of 3D observations, an easy substitution of traditional signal processing operators from images for processing unorganized 3D points is not possible. Furthermore, traditional estimators from classical statistics are not suitable for processing data in this domain, and new algorithms as well as different criteria for evaluating these algorithms are necessary. This dissertation contributes towards the development of two fundamental building blocks for processing point clouds. The first is of geometric model fitting, where we present a class of locally semi-parametric estimators that allows finite-sample analysis of accuracy and also explicitly addresses the problem of support-radius selection in local fitting. The second is of multi-scale filtering operators for point clouds that allow detection of interest regions whose locations as well as spatial extent are completely data-driven. The proposed approaches are distinguished from related work by operating directly in the input 3D space on unorganized points without assuming an available mesh or resorting to an intermediate global 2D parameterization. Results are presented for several applications including surface reconstruction, accurate shape descriptor computation and repeatable interest region detection, on synthetic data, as well as outdoor aerial and ground-based data obtained with a laser scanner.

[1]  Ronald Rosenfeld,et al.  Semi-supervised learning with graphs , 2005 .

[2]  Nicolas Le Roux,et al.  Learning Eigenfunctions Links Spectral Embedding and Kernel PCA , 2004, Neural Computation.

[3]  Li Yang.  Building k edge-disjoint spanning trees of minimum total length for isometric data embedding , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Frédo Durand,et al.  Non-iterative, feature-preserving mesh smoothing , 2003, ACM Trans. Graph..

[5]  Mark Meyer,et al.  Discrete Differential-Geometry Operators for Triangulated 2-Manifolds , 2002, VisMath.

[6]  Matthias Zwicker,et al.  Perspective Accurate Splatting , 2004, Graphics Interface.

[7]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Bernhard Schölkopf,et al.  Implicit Surfaces with Globally Regularised and Compactly Supported Basis Functions , 2006, NIPS.

[9]  Marc Pouget,et al.  Estimating differential quantities using polynomial fitting of osculating jets , 2003, Comput. Aided Geom. Des..

[10]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[11]  Mark Meyer,et al.  Implicit fairing of irregular meshes using diffusion and curvature flow , 1999, SIGGRAPH.

[12]  Brian C. Lovell,et al.  Algebraic Curve Fitting Support Vector Machines , 2003, DICTA.

[13]  Manuel Menezes de Oliveira Neto,et al.  Reconstructing manifold and non-manifold surfaces from point clouds , 2005, VIS 05. IEEE Visualization, 2005..

[14]  Nicolas Le Roux,et al.  Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering , 2003, NIPS.

[15]  Marc Levoy,et al.  QSplat: a multiresolution point rendering system for large meshes , 2000, SIGGRAPH.

[16]  Ko Nishino,et al.  Scale-Dependent 3D Geometric Features , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[17]  Markus H. Gross,et al.  Multiresolution Methods for Nonmanifold Models , 2001, IEEE Trans. Vis. Comput. Graph..

[18]  Kenichi Kanatani,et al.  Hyperaccuracy for Geometric Fitting , 2006 .

[19]  T. Hastie,et al.  Local Regression: Automatic Kernel Carpentry , 1993 .

[20]  W. Kahan,et al.  The Rotation of Eigenvectors by a Perturbation. III , 1970 .

[21]  Markus Schlattmann,et al.  Intrinsic Features on Surfaces , 2006 .

[22]  Leonidas J. Guibas,et al.  Estimating surface normals in noisy point cloud data , 2004, Int. J. Comput. Geom. Appl..

[23]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[24]  Martial Hebert,et al.  Analysis and Removal of Artifacts in 3-D LADAR Data , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[25]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[26]  Ali Shokoufandeh,et al.  Extracting 3D Shape Features in Discrete Scale-Space , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[27]  Guoliang Xu,et al.  Convergent discrete Laplace-Beltrami operators over triangular surfaces , 2004, Geometric Modeling and Processing, 2004. Proceedings.

[28]  Tony DeRose,et al.  Surface reconstruction from unorganized points , 1992, SIGGRAPH.

[29]  Matthew P. Wand,et al.  Kernel Smoothing , 1995 .

[30]  In-Kwon Lee,et al.  Curve reconstruction from unorganized points , 2000, Comput. Aided Geom. Des..

[31]  David Eppstein,et al.  A Deterministic Linear Time Algorithm for Geometric Separators and its Applications , 1995, Fundam. Informaticae.

[32]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[33]  Ramani Duraiswami,et al.  The improved fast Gauss transform with applications to machine learning , 2005 .

[34]  Herbert Edelsbrunner,et al.  Three-dimensional alpha shapes , 1992, VVS.

[35]  Richard Szeliski,et al.  Surface modeling with oriented particle systems , 1992, SIGGRAPH.

[36]  Ulrike von Luxburg,et al.  From Graphs to Manifolds - Weak and Strong Pointwise Consistency of Graph Laplacians , 2005, COLT.

[37]  Thomas Lewiner,et al.  Curvature and torsion estimators based on parametric curve fitting , 2005, Comput. Graph..

[38]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[39]  Matthias Zwicker,et al.  Surfels: surface elements as rendering primitives , 2000, SIGGRAPH.

[40]  Robert C. Reilly,et al.  Mean Curvature, the Laplacian, and Soap Bubbles , 1982 .

[41]  Mariette Yvinec,et al.  Delaunay Triangulation Based Surface Reconstruction : a short survey , 2004 .

[42]  Miguel Á. Carreira-Perpiñán,et al.  Proximity Graphs for Clustering and Manifold Learning , 2004, NIPS.

[43]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[44]  David Eppstein.  Offline Algorithms for Dynamic Minimum Spanning Tree Problems , 1994, J. Algorithms.

[45]  Gérard G. Medioni,et al.  Inference of Integrated Surface, Curve, and Junction Descriptions From Sparse 3D Data , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  William H. Press,et al.  Numerical recipes in C , 2002 .

[47]  Gabriel Taubin,et al.  A signal processing approach to fair surface design , 1995, SIGGRAPH.

[48]  Martial Hebert,et al.  Large data sets and confusing scenes in 3-D surface matching and recognition , 1999, Second International Conference on 3-D Digital Imaging and Modeling (Cat. No.PR00062).

[49]  J. Bunch,et al.  Updating the singular value decomposition , 1978 .

[50]  J. Koenderink.  The structure of images , 2004, Biological Cybernetics.

[51]  Pietro Perona,et al.  Object class recognition by unsupervised scale-invariant learning , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[52]  Marc Alexa,et al.  Point set surfaces , 2001, Proceedings Visualization, 2001. VIS '01..

[53]  Tosio Kato.  Perturbation theory for linear operators , 1966 .

[54]  Yongmin Li,et al.  On incremental and robust subspace learning , 2004, Pattern Recognit..

[55]  Holger Wendland,et al.  Scattered Data Approximation: Conditionally positive definite functions , 2004 .

[56]  Marc Alexa,et al.  Approximating and Intersecting Surfaces from Points , 2003, Symposium on Geometry Processing.

[57]  B. Triggs.  A New Approach to Geometric Fitting , 1996 .

[58]  V. Raykar,et al.  Fast large scale Gaussian process regression using approximate matrix-vector products , 2006 .

[59]  Wojciech Chojnacki,et al.  FNS, CFNS and HEIV: A Unifying Approach , 2005, Journal of Mathematical Imaging and Vision.

[60]  Peter Meer,et al.  Estimation of Nonlinear Errors-in-Variables Models for Computer Vision Applications , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Martial Hebert,et al.  Rapid object indexing using locality sensitive hashing and joint 3D-signature space estimation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  S. Shankar Sastry,et al.  Generalized principal component analysis (GPCA) , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[63]  Steven W. Zucker,et al.  Diffusion Maps and Geometric Harmonics for Automatic Target Recognition (ATR). Volume 2. Appendices , 2007 .

[64]  Marc Alexa,et al.  Anisotropic point set surfaces , 2006, AFRIGRAPH '06.

[65]  G. Stewart,et al.  Matrix Perturbation Theory , 1990 .

[66]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[67]  David G. Kirkpatrick,et al.  On the shape of a set of points in the plane , 1983, IEEE Trans. Inf. Theory.

[68]  T. Banchoff,et al.  Differential Geometry of Curves and Surfaces , 2010 .

[69]  Marc Levoy,et al.  The Use of Points as a Display Primitive , 2000 .

[70]  Larry S. Davis,et al.  Efficient Kernel Machines Using the Improved Fast Gauss Transform , 2004, NIPS.

[71]  D. Levin,et al.  Mesh-Independent Surface Interpolation , 2004 .

[72]  Bernt Schiele,et al.  3D object recognition from range images using local feature histograms , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[73]  I. Hassan.  Embedded , 2005, The Cyber Security Handbook.

[74]  Larry S. Davis,et al.  Efficient Kernel Density Estimation Using the Fast Gauss Transform with Applications to Color Modeling and Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[75]  Tony Lindeberg,et al.  Feature Detection with Automatic Scale Selection , 1998, International Journal of Computer Vision.

[76]  Sunghee Choi,et al.  A simple algorithm for homeomorphic surface reconstruction , 2000, SCG '00.

[77]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[78]  K. E. Barrett.  Degenerate polynomial forms , 1999 .

[79]  Dirk Langer,et al.  Imaging Ladar for 3-D Surveying and CAD Modeling of Real-World Environments , 2000, Int. J. Robotics Res..

[80]  Michael I. Jordan,et al.  Link Analysis, Eigenvectors and Stability , 2001, IJCAI.

[81]  Igor Guskov,et al.  Multi-scale features for approximate alignment of point-based surfaces , 2005, SGP '05.

[82]  Marshall W. Bern,et al.  A new Voronoi-based surface reconstruction algorithm , 1998, SIGGRAPH.

[83]  Matthias Zwicker,et al.  Pointshop 3D: an interactive system for point-based surface editing , 2002, SIGGRAPH.

[84]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[85]  Markus H. Gross,et al.  Efficient simplification of point-sampled surfaces , 2002, IEEE Visualization, 2002. VIS 2002..

[86]  Ji-guang Sun,et al.  Perturbation analysis for the generalized eigenvalue and the generalized singular value problem , 1983 .

[87]  Gene H. Golub,et al.  Optimal Surface Smoothing as Filter Design , 1996, ECCV.

[88]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[89]  Atsushi Imiya,et al.  Linear Scale-Space has First been Proposed in Japan , 1999, Journal of Mathematical Imaging and Vision.

[90]  Sunil Arya,et al.  An optimal algorithm for approximate nearest neighbor searching fixed dimensions , 1998, JACM.

[91]  Stefan Gumhold,et al.  Feature Extraction From Point Clouds , 2001, IMR.

[92]  Markus H. Gross,et al.  Spectral processing of point-sampled geometry , 2001, SIGGRAPH.

[93]  Gene H. Golub,et al.  Methods for modifying matrix factorizations , 1972, Milestones in Matrix Computation.

[94]  I. Holopainen.  Riemannian Geometry , 1927, Nature.

[95]  Leif Kobbelt,et al.  A survey of point-based techniques in computer graphics , 2004, Comput. Graph..

[96]  Gérard G. Medioni,et al.  First order augmentation to tensor voting for boundary inference and multiscale analysis in 3D , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[97]  Jitendra Malik,et al.  Recognizing Objects in Range Data Using Regional Point Descriptors , 2004, ECCV.

[98]  Markus H. Gross,et al.  Multi‐scale Feature Extraction on Point‐Sampled Surfaces , 2003, Comput. Graph. Forum.

[99]  Leif Kobbelt,et al.  Phong Splatting , 2004, PBG.

[100]  Anne Verroust-Blondet,et al.  Extracting skeletal curves from 3D scattered data , 1999, Proceedings Shape Modeling International '99. International Conference on Shape Modeling and Applications.

[101]  Tamal K. Dey,et al.  Provable surface reconstruction from noisy samples , 2006, Comput. Geom..

[102]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[103]  David P. Dobkin,et al.  A search engine for 3D models , 2003, TOGS.

[104]  Martial Hebert,et al.  Natural terrain classification using 3-d ladar data , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[105]  O. Sorkine.  Differential Representations for Mesh Processing , 2006 .

[106]  Ramesh C. Jain,et al.  Invariant surface characteristics for 3D object recognition in range images , 1985, Comput. Vis. Graph. Image Process..

[107]  David Mumford,et al.  Statistics of range images , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[108]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[109]  Adam Krzyzak,et al.  Learning and Design of Principal Curves , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[110]  Larry Wasserman,et al.  All of Statistics , 2004 .

[111]  Andrew P. Witkin,et al.  Scale-Space Filtering , 1983, IJCAI.

[112]  D. Cohen-Or,et al.  Robust moving least-squares fitting with sharp features , 2005, ACM Trans. Graph..

[113]  Szymon Rusinkiewicz,et al.  Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors , 2003, Symposium on Geometry Processing.

[114]  Martial Hebert,et al.  Scale Selection for the Analysis of Point-Sampled Curves , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[115]  Gabriel Taubin,et al.  An improved algorithm for algebraic curve and surface fitting , 1993, 1993 (4th) International Conference on Computer Vision.