ArchGANs: stylized colorization prototyping for architectural line drawing

Architectural illustration using line drawing with colorization is an important tool and art format. In this paper, in order to generate a natural-looking and high quality watercolorlike colorization for architectural line drawing, we propose a novel Generative Adversarial Network (GAN) approach, namely ArchGANs. The proposed ArchGANs unifies a line-feature-aware stylized colorization network (ArchColGAN), which can learn, predict and generate the coloring based on a dataset, as well as a shading generation network (ArchShdGAN), which augments the illustration with controllable lighting effects for better depicting building in 3D. Specifically, ArchColGAN can preserve the essential line features and building part correlation property, it also tackles the uneven colorization problem caused by the sparse lines. Experimental results demonstrate our proposed method is effective and suitable for colorization prototyping.

[1]  Aaron Hertzmann,et al.  A survey of stroke-based rendering , 2003, IEEE Computer Graphics and Applications.

[2]  Jing Liao,et al.  CariGANs , 2018, ACM Trans. Graph..

[3]  Ping Tan,et al.  DualGAN: Unsupervised Dual Learning for Image-to-Image Translation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[4]  Gang Hua,et al.  Visual attribute transfer through deep image analogy , 2017, ACM Trans. Graph..

[5]  Frédo Durand,et al.  Apparent ridges for line drawing , 2007, ACM Trans. Graph..

[6]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[7]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[9]  Eunhyeok Park,et al.  Tag2Pix: Line Art Colorization Using Text Tag With SECat and Changing Loss , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[10]  Edward Robbins The Art of Architectural Drawing: Imagination and Technique , 1997 .

[11]  Jan Kautz,et al.  Unsupervised Image-to-Image Translation Networks , 2017, NIPS.

[12]  Oliver Deussen,et al.  Watercolor Illustrations of CAD Data , 2008, CAe.

[13]  David Salesin,et al.  Computer-generated watercolor , 1997, SIGGRAPH.

[14]  Nenghai Yu,et al.  Coherent Online Video Style Transfer , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[15]  Jan Kautz,et al.  Multimodal Unsupervised Image-to-Image Translation , 2018, ECCV.

[16]  Chun-Fa Chang,et al.  Real-Time Rendering of Watercolor Effects for Virtual Environments , 2004, PCM.

[17]  Nenghai Yu,et al.  StyleBank: An Explicit Representation for Neural Image Style Transfer , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Frank Van Reeth,et al.  Real-time simulation of watery paint: Natural Phenomena and Special Effects , 2005 .

[19]  Oliver Deussen,et al.  Real-time watercolor illustrations of plants using a blurred depth test , 2006, NPAR.

[20]  Adam Finkelstein,et al.  Suggestive contours for conveying shape , 2003, ACM Trans. Graph..

[21]  Yoshua Bengio,et al.  Generative Adversarial Networks , 2014, ArXiv.

[22]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[24]  Adrien Bousseau,et al.  Interactive watercolor rendering with temporal coherence and abstraction , 2006, NPAR.

[25]  Hans-Peter Seidel,et al.  Ridge-valley lines on meshes via implicit surface fitting , 2004, ACM Trans. Graph..

[26]  Chiew-Lan Tai,et al.  MoXi: real-time ink dispersion in absorbent paper , 2005, SIGGRAPH '05.

[27]  Hyunsoo Kim,et al.  Learning to Discover Cross-Domain Relations with Generative Adversarial Networks , 2017, ICML.

[28]  Alexei A. Efros,et al.  Toward Multimodal Image-to-Image Translation , 2017, NIPS.

[29]  Ilan Shimshoni,et al.  On edge detection on surfaces , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.