Mesh denoising via cascaded normal regression

We present a data-driven approach for mesh denoising. Our key idea is to formulate the denoising process with cascaded non-linear regression functions and learn them from a set of noisy meshes and their ground-truth counterparts. Each regression function infers the normal of a denoised output mesh facet from geometry features extracted from its neighborhood facets on the input mesh and sends the result as the input of the next regression function. Specifically, we develop a filtered facet normal descriptor (FND) for modeling the geometry features around each facet on the noisy mesh and model a regression function with neural networks for mapping the FNDs to the facet normals of the denoised mesh. To handle meshes with different geometry features and reduce the training difficulty, we cluster the input mesh facets according to their FNDs and train neural networks for each cluster separately in an offline learning stage. At runtime, our method applies the learned cascaded regression functions to a noisy input mesh and reconstructs the denoised mesh from the output facet normals. Our method learns the non-linear denoising process from the training data and makes no specific assumptions about the noise distribution and geometry features in the input. The runtime denoising process is fully automatic for different input meshes. Our method can be easily adapted to meshes with arbitrary noise patterns by training a dedicated regression scheme with mesh data and the particular noise pattern. We evaluate our method on meshes with both synthetic and real scanned noise, and compare it to other mesh denoising algorithms. Results demonstrate that our method outperforms the state-of-the-art mesh denoising methods and successfully removes different kinds of noise for meshes with various geometry features.

[1]  Yizhou Yu,et al.  Robust Feature-Preserving Mesh Denoising Based on Consistent Subneighborhoods , 2010, IEEE Transactions on Visualization and Computer Graphics.

[2]  Andreas Kolb,et al.  Kinect range sensing: Structured-light versus Time-of-Flight Kinect , 2015, Comput. Vis. Image Underst..

[3]  Keenan Crane,et al.  A General Framework for Bilateral and Mean Shift Filtering , 2014, ArXiv.

[4]  Gabriel Taubin,et al.  A signal processing approach to fair surface design , 1995, SIGGRAPH.

[5]  Sebastian Nowozin,et al.  Cascades of Regression Tree Fields for Image Restoration , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Renjie Liao,et al.  Deep Edge-Aware Filters , 2015, ICML.

[7]  Daniel Cohen-Or,et al.  Bilateral mesh denoising , 2003 .

[8]  Jian Sun,et al.  Cascaded hand pose regression , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Konrad Polthier,et al.  Anisotropic Filtering of Non‐Linear Surface Features , 2004, Comput. Graph. Forum.

[10]  Bochang Moon,et al.  Adaptive Rendering Based on Weighted Local Regression , 2014, ACM Trans. Graph..

[11]  Kun Zhou,et al.  Displaced dynamic expression regression for real-time facial tracking and animation , 2014, ACM Trans. Graph..

[12]  Aaron Hertzmann,et al.  Learning 3D mesh segmentation and labeling , 2010, SIGGRAPH 2010.

[13]  Zhigang Deng,et al.  A Robust Scheme for Feature-Preserving Mesh Denoising , 2016, IEEE Transactions on Visualization and Computer Graphics.

[14]  Pietro Perona,et al.  Cascaded pose regression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Youyi Zheng,et al.  IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 1 Bilateral Normal Filtering for Mesh Denoising , 2022 .

[16]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Kenny Mitchell,et al.  Adaptive rendering with linear predictions , 2015, ACM Trans. Graph..

[18]  Martin Rumpf,et al.  Anisotropic geometric diffusion in surface processing , 2000 .

[19]  Donghoon Lee,et al.  Face alignment using cascade Gaussian process regression trees , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Bernard Widrow,et al.  Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[21]  Bailin Deng,et al.  Guided Mesh Normal Filtering , 2015, Comput. Graph. Forum.

[22]  Pradeep Sen,et al.  A machine learning approach for filtering Monte Carlo noise , 2015, ACM Trans. Graph..

[23]  Ralph R. Martin,et al.  Noise analysis and synthesis for 3D laser depth scanners , 2009, Graph. Model..

[24]  Yutaka Ohtake,et al.  Mesh denoising via iterative alpha-trimming and nonlinear diffusion of normals with automatic thresholding , 2003, Proceedings Computer Graphics International 2003.

[25]  Ligang Liu,et al.  Bi-Normal Filtering for Mesh Denoising , 2015, IEEE Transactions on Visualization and Computer Graphics.

[26]  Martin Rumpf,et al.  Anisotropic geometric diffusion in surface processing , 2000, Proceedings Visualization 2000. VIS 2000 (Cat. No.00CH37145).

[27]  Antonio Criminisi,et al.  Filter Forests for Learning Data-Dependent Convolutional Kernels , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Sebastian Nowozin,et al.  Interleaved Regression Tree Field Cascades for Blind Image Deconvolution , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[29]  R. Horaud,et al.  Surface feature detection and description with applications to mesh matching , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Ligang Liu,et al.  Decoupling noise and features via weighted ℓ1-analysis compressed sensing , 2014, TOGS.

[31]  Yutaka Ohtake,et al.  Mesh smoothing via mean and median filtering applied to face normals , 2002, Geometric Modeling and Processing. Theory and Applications. GMP 2002. Proceedings.

[32]  Mark Meyer,et al.  Implicit fairing of irregular meshes using diffusion and curvature flow , 1999, SIGGRAPH.

[33]  Partha Pratim Das,et al.  Characterizations of Noise in Kinect Depth Images: A Review , 2014, IEEE Sensors Journal.

[34]  Dani Lischinski,et al.  Joint bilateral upsampling , 2007, ACM Trans. Graph..

[35]  Ross T. Whitaker,et al.  Geometric surface smoothing via anisotropic diffusion of normals , 2002, IEEE Visualization, 2002. VIS 2002..

[36]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[37]  Lei He,et al.  Mesh denoising via L0 minimization , 2013, ACM Trans. Graph..

[38]  Sebastian Thrun,et al.  A Bayesian method for probable surface reconstruction and decimation , 2006, TOGS.

[39]  Dani Lischinski,et al.  Joint bilateral upsampling , 2007, SIGGRAPH 2007.

[40]  Yuzhong Shen,et al.  Fuzzy vector median-based surface smoothing , 2004, IEEE Transactions on Visualization and Computer Graphics.

[41]  Andrew Adams,et al.  Fast High‐Dimensional Filtering Using the Permutohedral Lattice , 2010, Comput. Graph. Forum.

[42]  Jian Sun,et al.  Joint Cascade Face Detection and Alignment , 2014, ECCV.

[43]  Wenping Wang,et al.  Feature-preserving mesh denoising via bilateral normal filtering , 2005, Ninth International Conference on Computer Aided Design and Computer Graphics (CAD-CG'05).

[44]  Radu Horaud,et al.  Surface feature detection and description with applications to mesh matching , 2009, CVPR.

[45]  Frédo Durand,et al.  Non-iterative, feature-preserving mesh smoothing , 2003, ACM Trans. Graph..

[46]  Paul Suetens,et al.  Feature detection on 3D face surfaces for pose normalisation and recognition , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[47]  Chandrajit L. Bajaj,et al.  Anisotropic diffusion of surfaces and functions on surfaces , 2003, TOGS.

[48]  Alexandre Boulch,et al.  Deep Learning for Robust Normal Estimation in Unstructured Point Clouds , 2016, Comput. Graph. Forum.

[49]  Stefan Harmeling,et al.  Image denoising: Can plain neural networks compete with BM3D? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Ralph R. Martin,et al.  Fast and Effective Feature-Preserving Mesh Denoising , 2007, IEEE Transactions on Visualization and Computer Graphics.

[51]  Aaron Hertzmann,et al.  Learning 3D mesh segmentation and labeling , 2010, ACM Trans. Graph..