Deep point correlation design

Designing point patterns with desired properties can require substantial effort, both in hand-crafting coding and mathematical derivation. Retaining these properties in multiple dimensions or for a substantial number of points can be challenging and computationally expensive. Tackling those two issues, we suggest to automatically generate scalable point patterns from design goals using deep learning. We phrase pattern generation as a deep composition of weighted distance-based unstructured filters. Deep point pattern design means to optimize over the space of all such compositions according to a user-provided point correlation loss, a small program which measures a pattern's fidelity in respect to its spatial or spectral statistics, linear or non-linear (e. g., radial) projections, or any arbitrary combination thereof. Our analysis shows that we can emulate a large set of existing patterns (blue, green, step, projective, stair, etc.-noise), generalize them to countless new combinations in a systematic way and leverage existing error estimation formulations to generate novel point patterns for a user-provided class of integrand functions. Our point patterns scale favorably to multiple dimensions and numbers of points: we demonstrate nearly 10k points in 10-D produced in one second on one GPU. All the resources (source code and the pre-trained networks) can be found at https://sampling.mpi-inf.mpg.de/deepsampling.html.

[1]  Jian-Jun Zhang,et al.  Blue noise sampling using an SPH-based method , 2015, ACM Trans. Graph..

[2]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[3]  Justin Solomon,et al.  Stochastic Wasserstein Barycenters , 2018, ICML.

[4]  Jennifer Werfel,et al.  Orthogonal Arrays Theory And Applications , 2016 .

[5]  Timo Ropinski,et al.  Monte Carlo convolution for learning on non-uniformly sampled point clouds , 2018, ACM Trans. Graph..

[6]  Li-Yi Wei,et al.  Point sampling with general noise spectrum , 2012, ACM Trans. Graph..

[7]  Robert Ulichney,et al.  Dithering with blue noise , 1988, Proc. IEEE.

[8]  Li-Yi Wei,et al.  Differential domain analysis for non-uniform sampling , 2011, ACM Trans. Graph..

[9]  Yi Chen,et al.  Wasserstein blue noise sampling , 2017, TOGS.

[10]  Gurprit Singh,et al.  Fast tile-based adaptive sampling with user-specified Fourier spectra , 2014, ACM Trans. Graph..

[11]  Gurprit Singh,et al.  Analysis of Sample Correlations for Monte Carlo Rendering , 2019, Comput. Graph. Forum.

[12]  Carola Doerr,et al.  Constructing low star discrepancy point sets with genetic algorithms , 2013, GECCO '13.

[13]  Paul S. Heckbert,et al.  Graphics gems IV , 1994 .

[14]  Peter Shirley,et al.  Multi-Jittered Sampling , 1994, Graphics Gems.

[15]  Samy Bengio,et al.  Density estimation using Real NVP , 2016, ICLR.

[16]  Ligang Liu,et al.  Variational Blue Noise Sampling , 2012, IEEE Transactions on Visualization and Computer Graphics.

[17]  Oliver Deussen,et al.  Blue noise sampling with controlled aliasing , 2013, TOGS.

[18]  Peter Shirley,et al.  Discrepancy as a Quality Measure for Sample Distributions , 1991, Eurographics.

[19]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[20]  Clifford Stein,et al.  Sony Pictures Imageworks Arnold , 2018, ACM Trans. Graph..

[21]  Pat Hanrahan,et al.  Sequences with Low‐Discrepancy Blue‐Noise 2‐D Projections , 2018, Comput. Graph. Forum.

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[23]  Zhi-Hua Zhou,et al.  A brief introduction to weakly supervised learning , 2018 .

[24]  D. W. Scott On optimal and data based histograms , 1979 .

[25]  Greg Humphreys,et al.  Physically Based Rendering: From Theory to Implementation , 2004 .

[26]  David Eppstein,et al.  Computing the discrepancy with applications to supersampling patterns , 1996, TOGS.

[27]  Adrian Secord,et al.  Weighted Voronoi stippling , 2002, NPAR '02.

[28]  Raquel Urtasun,et al.  Deep Parametric Continuous Convolutional Neural Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Andrew Kensler,et al.  Orthogonal Array Sampling for Monte Carlo Rendering , 2019, Comput. Graph. Forum.

[30]  Raanan Fattal Blue-noise point sampling using kernel density model , 2011, SIGGRAPH 2011.

[31]  Frances Y. Kuo,et al.  Constructing Sobol Sequences with Better Two-Dimensional Projections , 2008, SIAM J. Sci. Comput..

[32]  Pankaj K. Agarwal,et al.  Geometric Range Searching and Its Relatives , 2007 .

[33]  Benoit B. Mandelbrot,et al.  Fractal Geometry of Nature , 1984 .

[34]  Abdalla G. M. Ahmed,et al.  Low-discrepancy blue noise sampling , 2016, ACM Trans. Graph..

[35]  Matthias Zwicker,et al.  Learning to Importance Sample in Primary Sample Space , 2018, Comput. Graph. Forum.

[36]  Albert J. Ahumada,et al.  Principled halftoning based on human vision models , 1992, Electronic Imaging.

[37]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  I. Sloan Lattice Methods for Multiple Integration , 1994 .

[39]  Yaron Lipman,et al.  Point convolutional neural networks by extension operators , 2018, ACM Trans. Graph..

[40]  Don P. Mitchell,et al.  Ray Tracing and Irregularities of Distribution , 2000 .

[41]  Pramod K. Varshney,et al.  Stair blue noise sampling , 2016, ACM Trans. Graph..

[42]  J. Yellott Spectral consequences of photoreceptor sampling in the rhesus retina. , 1983, Science.

[43]  Dani Lischinski,et al.  Recursive Wang tiles for real-time blue noise , 2006, ACM Trans. Graph..

[44]  A. Cengiz Öztireli Integration with Stochastic Point Processes , 2016, ACM Trans. Graph..

[45]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[46]  Hans-Peter Seidel,et al.  Deep Shading: Convolutional Neural Networks for Screen Space Shading , 2016, Comput. Graph. Forum.

[47]  H. Niederreiter Quasi-Monte Carlo methods and pseudo-random numbers , 1978 .

[48]  Markus Gross,et al.  Analysis and synthesis of point distributions based on pair correlation , 2012, ACM Trans. Graph..

[49]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[50]  S. Dammertz,et al.  Image Synthesis by Rank-1 Lattices , 2008 .

[51]  Markus H. Gross,et al.  Deep scattering , 2017, ACM Trans. Graph..

[52]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Leonidas J. Guibas,et al.  ShapeNet: An Information-Rich 3D Model Repository , 2015, ArXiv.

[54]  J. S. Hicks,et al.  An efficient method for generating uniformly distributed points on the surface of an n-dimensional sphere , 1959, CACM.

[55]  Harald Niederreiter,et al.  Random number generation and Quasi-Monte Carlo methods , 1992, CBMS-NSF regional conference series in applied mathematics.

[56]  S. Hubbell,et al.  Spatial patterns in the distribution of tropical tree species. , 2000, Science.

[57]  Li-Yi Wei,et al.  Parallel Poisson disk sampling with spectrum analysis on surfaces , 2010, ACM Trans. Graph..

[58]  I. Sobol On the distribution of points in a cube and the approximate evaluation of integrals , 1967 .

[59]  Bin Wang,et al.  A Survey of Blue-Noise Sampling and Its Applications , 2015, Journal of Computer Science and Technology.

[60]  Alexander Keller,et al.  Learning light transport the reinforced way , 2016, SIGGRAPH Talks.

[61]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[62]  Ares Lagae,et al.  A Comparison of Methods for Generating Poisson Disk Distributions , 2008, Comput. Graph. Forum.

[63]  Gurprit Singh,et al.  Fourier Analysis of Correlated Monte Carlo Importance Sampling , 2019, Comput. Graph. Forum.

[64]  Philip Levis,et al.  Ebb: A DSL for Physical Simluation on CPUs and GPUs , 2015, ACM Trans. Graph..

[65]  Andrew Kensler,et al.  Progressive Multi‐Jittered Sample Sequences , 2018, Comput. Graph. Forum.

[66]  Robert L. Cook,et al.  Stochastic sampling in computer graphics , 1988, TOGS.

[67]  Mathieu Desbrun,et al.  Blue noise through optimal transport , 2012, ACM Trans. Graph..

[68]  Timo Aila,et al.  Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder , 2017, ACM Trans. Graph..

[69]  A. Owen Monte Carlo Variance of Scrambled Net Quadrature , 1997 .

[70]  Shakir Mohamed,et al.  Variational Inference with Normalizing Flows , 2015, ICML.

[71]  Mark Meyer,et al.  Kernel-predicting convolutional networks for denoising Monte Carlo renderings , 2017, ACM Trans. Graph..

[72]  Leonardo Colzani,et al.  Mean square decay of Fourier transforms in Euclidean and non Euclidean spaces , 2001 .

[73]  Eero P. Simoncelli,et al.  Natural image statistics and neural representation. , 2001, Annual review of neuroscience.

[74]  Dirk Nuyens The construction of good lattice rules and polynomial lattice rules , 2014, Uniform Distribution and Quasi-Monte Carlo Methods.

[75]  Alexander Keller,et al.  Advanced (quasi) Monte Carlo methods for image synthesis , 2012, SIGGRAPH '12.

[76]  Jan Kautz,et al.  Fourier analysis of stochastic sampling strategies for assessing bias and variance in integration , 2013, ACM Trans. Graph..

[77]  Raanan Fattal,et al.  Blue-noise point sampling using kernel density model , 2011, ACM Trans. Graph..

[78]  V. Ostromoukhov,et al.  Fast hierarchical importance sampling with blue noise properties , 2004, SIGGRAPH 2004.

[79]  Oliver Deussen,et al.  Floating Points: A Method for Computing Stipple Drawings , 2000, Comput. Graph. Forum.

[80]  Mohamed S. Ebeida,et al.  Spoke-Darts for High-Dimensional Blue-Noise Sampling , 2014, ACM Trans. Graph..

[81]  Thomas Müller,et al.  Neural Importance Sampling , 2018, ACM Trans. Graph..

[82]  Joachim Weickert,et al.  Universität Des Saarlandes Fachrichtung 6.1 – Mathematik Electrostatic Halftoning Electrostatic Halftoning , 2022 .

[83]  Matthias Nießner,et al.  Opt , 2016, ACM Trans. Graph..

[84]  Gurprit Singh,et al.  Variance analysis for Monte Carlo integration , 2015, ACM Trans. Graph..

[85]  Hans-Peter Seidel,et al.  Projective Blue‐Noise Sampling , 2016, Comput. Graph. Forum.

[86]  J. Halton On the efficiency of certain quasi-random sequences of points in evaluating multi-dimensional integrals , 1960 .

[87]  Edward H. Adelson,et al.  Statistics of real-world illumination , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[88]  Marcos Fajardo,et al.  Blue-noise dithered sampling , 2016, SIGGRAPH Talks.

[89]  Joachim Weickert,et al.  Fast electrostatic halftoning , 2011, Journal of Real-Time Image Processing.

[90]  Frédo Durand,et al.  Differentiable programming for image processing and deep learning in halide , 2018, ACM Trans. Graph..

[91]  Gurprit Singh,et al.  Convergence analysis for anisotropic monte carlo sampling spectra , 2017, ACM Trans. Graph..

[92]  Eugene Fiume,et al.  Hierarchical Poisson disk sampling distributions , 1992 .

[93]  Peiran REN,et al.  Image based relighting using neural networks , 2015, ACM Trans. Graph..

[94]  Pradeep Sen,et al.  A machine learning approach for filtering Monte Carlo noise , 2015, ACM Trans. Graph..

[95]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[96]  Jaakko Lehtinen,et al.  Aether , 2017, ACM Trans. Graph..

[97]  Gordon Wetzstein,et al.  ProxImaL , 2016, ACM Trans. Graph..

[98]  Michael Balzer,et al.  Capacity-constrained point distributions: a variant of Lloyd's method , 2009, ACM Trans. Graph..