Computational Parquetry

Parquetry is the art and craft of decorating a surface with a pattern of differently colored veneers of wood, stone, or other materials. Traditionally, the process of designing and making parquetry has been driven by color, using the texture found in real wood only for stylization or as a decorative effect. Here, we introduce a computational pipeline that draws from the rich natural structure of strongly textured real-world veneers as a source of detail to approximate a target image as faithfully as possible using a manageable number of parts. This challenge is closely related to the established problems of patch-based image synthesis and stylization in some ways, but fundamentally different in others. Most importantly, the limited availability of resources (any piece of wood can only be used once) turns the relatively simple problem of finding the right piece for the target location into the combinatorial problem of finding optimal parts while avoiding resource collisions. We introduce an algorithm that efficiently solves an approximation to the problem. It further addresses challenges like gamut mapping, feature characterization, and the search for fabricable cuts. We demonstrate the effectiveness of the system by fabricating a selection of pieces of parquetry from different kinds of unstained wood veneer.

[1]  Nikolay Jetchev,et al.  GANosaic: Mosaic Creation with Generative Texture Manifolds , 2017, ArXiv.

[2]  William T. Freeman,et al.  The patch transform and its applications to image editing , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Adam Finkelstein,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, SIGGRAPH 2009.

[4]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[5]  Marc Levoy,et al.  Fast texture synthesis using tree-structured vector quantization , 2000, SIGGRAPH.

[6]  Stephen DiVerdi,et al.  RealBrush: painting with examples of physical media , 2013, ACM Trans. Graph..

[7]  Denis Simakov,et al.  Summarizing visual data using bidirectional similarity , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Jian Sun,et al.  Computing nearest-neighbor fields via Propagation-Assisted KD-Trees , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and environment maps , 1997, SIGGRAPH.

[10]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Yael Pritch,et al.  Shift-map image editing , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[12]  Eli Shechtman,et al.  Image melding , 2012, ACM Trans. Graph..

[13]  Qi Zhang,et al.  Rolling Guidance Filter , 2014, ECCV.

[14]  Chuan Li,et al.  Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Yu Liu,et al.  Generating Classic Mosaics with Graph Cuts , 2010, Comput. Graph. Forum.

[16]  Frederic Fol Leymarie,et al.  Sketches by Paul the robot , 2012, CAe '12.

[17]  Youngsup Park,et al.  Mosaic Rendering using Colored Paper , 2006, VAST.

[18]  PellaciniFabio,et al.  Jigsaw image mosaics , 2002 .

[19]  Shai Avidan,et al.  TreeCANN - k-d Tree Coherence Approximate Nearest Neighbor Algorithm , 2012, ECCV.

[20]  Michael Ashikhmin,et al.  Synthesizing natural textures , 2001, I3D '01.

[21]  Oliver Deussen,et al.  Feedback-guided stroke placement for a painting machine , 2012, CAe '12.

[22]  Sebastiano Battiato,et al.  Artificial Mosaics with Irregular Tiles Based on Gradient Vector Flow , 2012, ECCV Workshops.

[23]  Daniel P. Huttenlocher,et al.  Distance Transforms of Sampled Functions , 2012, Theory Comput..

[24]  Jitendra Malik,et al.  Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons , 2001, International Journal of Computer Vision.

[25]  Dani Lischinski,et al.  Solid texture synthesis from 2D exemplars , 2007, ACM Trans. Graph..

[26]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[27]  Nipun Kwatra,et al.  Texture optimization for example-based synthesis , 2005, ACM Trans. Graph..

[28]  Kun Zhou,et al.  Fast example-based surface texture synthesis via discrete optimization , 2006, The Visual Computer.

[29]  Gershon Elber,et al.  Rendering traditional mosaics , 2003, The Visual Computer.

[30]  Sylvain Lefebvre,et al.  State of the Art in Example-based Texture Synthesis , 2009, Eurographics.

[31]  Tobias Isenberg,et al.  State of the "Art”: A Taxonomy of Artistic Stylization Techniques for Images and Video , 2013, IEEE Transactions on Visualization and Computer Graphics.

[32]  Rupert Paget,et al.  Texture synthesis via a noncausal nonparametric multiscale Markov random field , 1998, IEEE Trans. Image Process..

[33]  Oliver Deussen,et al.  Image stylization with a painting machine using semantic hints , 2013, Comput. Graph..

[34]  Irfan A. Essa,et al.  Graphcut textures: image and video synthesis using graph cuts , 2003, ACM Trans. Graph..

[35]  Giovanni Gallo,et al.  Puzzle Image Mosaic , 2005 .

[36]  Dani Lischinski,et al.  Self Tuning Texture Optimization , 2015, Comput. Graph. Forum.

[37]  Adam Finkelstein,et al.  The Generalized PatchMatch Correspondence Algorithm , 2010, ECCV.

[38]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and texture-mapped models , 1997, International Conference on Computer Graphics and Interactive Techniques.

[39]  Baining Guo,et al.  Synthesis of bidirectional texture functions on arbitrary surfaces , 2002, SIGGRAPH.

[40]  Kei Iwasaki,et al.  Fabricating reflectors for displaying multiple images , 2018, ACM Trans. Graph..

[41]  Alejo Hausner,et al.  Simulating decorative mosaics , 2001, SIGGRAPH.

[42]  G. D. Blasi,et al.  Fast Photomosaic , 2005 .

[43]  Emily Whiting,et al.  Watercolor Woodblock Printing with Image Analysis , 2018, Comput. Graph. Forum.

[44]  Huamin Wang,et al.  Wetbrush: GPU-based 3D painting simulation at the bristle level , 2015, ACM Trans. Graph..

[45]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[46]  Eli Shechtman,et al.  Space-Time Completion of Video , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Giovanni Maria Farinella,et al.  Digital Mosaic Frameworks ‐ An Overview , 2007, Comput. Graph. Forum.

[48]  Fabio Pellacini,et al.  Jigsaw image mosaics , 2002, ACM Trans. Graph..

[49]  Baining Guo,et al.  Real-time texture synthesis by patch-based sampling , 2001, TOGS.

[50]  Bernd Bickel,et al.  State of the Art on Stylized Fabrication , 2018, Comput. Graph. Forum.

[51]  Albert J. Oakes,et al.  The Complete Manual of Woodworking , 1989 .

[52]  Nicole Immorlica,et al.  Locality-sensitive hashing scheme based on p-stable distributions , 2004, SCG '04.

[53]  Eero P. Simoncelli,et al.  A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients , 2000, International Journal of Computer Vision.

[54]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[55]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Robert S. Lubar,et al.  Dali: The Salvador Dali Museum Collection , 1991 .

[57]  Adam Finkelstein,et al.  Lapped textures , 2000, SIGGRAPH.

[58]  Kyunghyun Yoon,et al.  A Parallel Framework for Fast Photomosaics , 2011, IEICE Trans. Inf. Syst..

[59]  Leif Kobbelt,et al.  GIzMOs: Genuine Image Mosaics with Adaptive Tiling , 2009, Comput. Graph. Forum.

[60]  Adam Finkelstein,et al.  Image Mosaics , 1998, EP.

[61]  Alvaro Soto,et al.  Human detection using a mobile platform and novel features derived from a visual saliency mechanism , 2010, Image Vis. Comput..

[62]  Leon A. Gatys,et al.  Texture Synthesis Using Convolutional Neural Networks , 2015, NIPS.

[63]  Zunlei Feng,et al.  Neural Style Transfer: A Review , 2017, IEEE Transactions on Visualization and Computer Graphics.

[64]  Fang-Lue Zhang,et al.  A survey of the state-of-the-art in patch-based synthesis , 2017, Computational Visual Media.

[65]  Alexei A. Efros,et al.  Texture synthesis by non-parametric sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[66]  Chuan Li,et al.  Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks , 2016, ECCV.

[67]  Giovanni Maria Farinella,et al.  A Survey of Digital Mosaic Techniques , 2006, Eurographics Italian Chapter Conference.

[68]  Adam Finkelstein,et al.  The PatchMatch randomized matching algorithm for image manipulation , 2011, Commun. ACM.