A Neural Representation of Sketch Drawings

We present sketch-rnn, a recurrent neural network (RNN) able to construct stroke-based drawings of common objects. The model is trained on thousands of crude human-drawn images representing hundreds of classes. We outline a framework for conditional and unconditional sketch generation, and describe new robust training methods for generating coherent sketch drawings in a vector format.

[1]  David H. Douglas,et al.  ALGORITHMS FOR THE REDUCTION OF THE NUMBER OF POINTS REQUIRED TO REPRESENT A DIGITIZED LINE OR ITS CARICATURE , 1973 .

[2]  C. Bishop.  Mixture density networks , 1994 .

[3]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[4]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[5]  Aaron J. Alford.  The Quick , 2008 .

[6]  Yong Jae Lee,et al.  ShadowDraw: real-time user guidance for freehand drawing , 2011, SIGGRAPH 2011.

[7]  Marc Alexa,et al.  How do humans sketch objects? , 2012, ACM Trans. Graph..

[8]  Masashi Sugiyama,et al.  Artist Agent: A Reinforcement Learning Approach to Automatic Stroke Generation in Oriental Ink Painting , 2012, ICML.

[9]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[10]  Frederic Fol Leymarie,et al.  Portrait drawing by Paul the robot , 2013, Comput. Graph..

[11]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[12]  David Ha,et al.  Recurrent Net Dreams Up Fake Chinese Characters in Vector Format with TensorFlow , 2015 .

[13]  Joshua B. Tenenbaum,et al.  Human-level concept learning through probabilistic program induction , 2015, Science.

[14]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[15]  Ole Winther,et al.  Ladder Variational Autoencoders , 2016, NIPS.

[16]  Tom White,et al.  Sampling Generative Networks: Notes on a Few Effective Techniques , 2016, ArXiv.

[17]  Geoffrey E. Hinton,et al.  Layer Normalization , 2016, ArXiv.

[18]  Erhardt Barth,et al.  Recurrent Dropout without Memory Loss , 2016, COLING.

[19]  Samy Bengio,et al.  Generating Sentences from a Continuous Space , 2015, CoNLL.

[20]  James Hays,et al.  The sketchy database , 2016, ACM Trans. Graph..

[21]  Ian J. Goodfellow,et al.  NIPS 2016 Tutorial: Generative Adversarial Networks , 2016, ArXiv.

[22]  Sergio Gomez Colmenarejo,et al.  Parallel Multiscale Autoregressive Density Estimation , 2017, ICML.

[23]  Quoc V. Le,et al.  HyperNetworks , 2016, ICLR.

[24]  Max Welling,et al.  Improved Variational Inference with Inverse Autoregressive Flow , 2016, NIPS 2016.

[25]  Hyunsoo Kim,et al.  Learning to Discover Cross-Domain Relations with Generative Adversarial Networks , 2017, ICML.

[26]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Jan Kautz,et al.  Unsupervised Image-to-Image Translation Networks , 2017, NIPS.

[28]  Yike Guo,et al.  Unsupervised Image-to-Image Translation with Generative Adversarial Networks , 2017, ArXiv.

[29]  Eric P. Xing,et al.  Controllable Text Generation , 2017, ArXiv.

[30]  Yoshua Bengio,et al.  Drawing and Recognizing Chinese Characters with Recurrent Neural Network , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.