Mobile Robotic Painting of Texture

Robotic painting is well-established in controlled factory environments, but there is now potential for mobile robots to do functional painting tasks around the everyday world. An obvious first target for such robots is painting a uniform single color. A step further is the painting of textured images. Texture involves a varying appearance, and requires that paint is delivered accurately onto the physical surface to produce the desired effect. Robotic painting of texture is relevant for architecture and in themed environments.A key challenge for robotic painting of texture is to take a desired image as input, and to generate the paint commands to as closely as possible create the desired appearance, according to the robotic capabilities. This paper describes a deep learning approach to take an input ink map of a desired texture, and infer robotic paint commands to produce that texture.We analyze the trade-offs between quality of reconstructed appearance and ease of execution. Our method is general for different kinds of robotic paint delivery systems, but the emphasis here is on spray painting. More generally, the framework can be viewed as an approach for solving a specific class of inverse imaging problems.

[1]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[2]  Nikos Komodakis,et al.  Wide Residual Networks , 2016, BMVC.

[3]  Yisong Yue,et al.  Iterative Amortized Inference , 2018, ICML.

[4]  Paolo Gallina,et al.  Watercolour Robotic Painting: a Novel Automatic System for Artistic Rendering , 2018, J. Intell. Robotic Syst..

[5]  R. J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[6]  Yisong Yue,et al.  Factorized Variational Autoencoders for Modeling Audience Reactions to Movies , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Sabine Süsstrunk,et al.  Fourier-Domain Optimization for Image Processing , 2018, ArXiv.

[8]  Frank Sehnke,et al.  Policy Gradients with Parameter-Based Exploration for Control , 2008, ICANN.

[9]  P. Kubelka,et al.  New contributions to the optics of intensely light-scattering materials. , 1954, Journal of the Optical Society of America.

[10]  Gary W. Meyer,et al.  Modeling pigmented materials for realistic image synthesis , 1992, TOGS.

[11]  Masashi Sugiyama,et al.  Artist Agent: A Reinforcement Learning Approach to Automatic Stroke Generation in Oriental Ink Painting , 2012, ICML.

[12]  Stephan Mandt,et al.  Quasi-Monte Carlo Variational Inference , 2018, ICML.

[13]  Rama Chellappa,et al.  Visual Domain Adaptation: A survey of recent advances , 2015, IEEE Signal Processing Magazine.

[14]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[16]  Zunlei Feng,et al.  Neural Style Transfer: A Review , 2017, IEEE Transactions on Visualization and Computer Graphics.

[17]  Paolo Gallina,et al.  Automatic Path-Planning Algorithm for Realistic Decorative Robotic Painting , 2013 .

[18]  Oliver Deussen,et al.  Feedback-guided stroke placement for a painting machine , 2012, CAe '12.

[19]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[20]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[21]  Jun Han,et al.  Deep Probabilistic Video Compression , 2018, ArXiv.

[22]  Corneliu Florea,et al.  Pandora: Description of a painting database for art movement recognition with baselines and perspectives , 2016, 2016 24th European Signal Processing Conference (EUSIPCO).

[23]  Gary W. Meyer,et al.  Airbrush simulation for artwork and computer modeling , 2009, NPAR '09.

[24]  B. Latha Shankar,et al.  Optimization of Robotic Spray Painting process Parameters using Taguchi Method , 2018 .

[25]  William V. Baxter,et al.  IMPaSTo: a realistic, interactive model for paint , 2004, NPAR '04.

[26]  Curtis R. Vogel,et al.  Ieee Transactions on Image Processing Fast, Robust Total Variation{based Reconstruction of Noisy, Blurred Images , 2022 .

[27]  Geoffrey E. Hinton,et al.  Inferring Motor Programs from Images of Handwritten Digits , 2005, NIPS.

[28]  Paolo Gallina,et al.  Airbrush Robotic Painting System: Experimental Validation of a Colour Spray Model , 2017, RAAD.

[29]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[30]  P. Kubelka,et al.  Errata: New Contributions to the Optics of Intensely Light-Scattering Materials. Part I , 1948 .

[31]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[32]  Anurag Sai Vempati,et al.  PaintCopter: An Autonomous UAV for Spray Painting on Three-Dimensional Surfaces , 2018, IEEE Robotics and Automation Letters.

[33]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[34]  Olga Sorkine-Hornung,et al.  Large-scale painting of photographs by interactive optimization , 2016, Comput. Graph..

[35]  Ren C. Luo,et al.  Robot Artist for colorful picture painting with visual control system , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[36]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[37]  Dit-Yan Yeung,et al.  Learning a Deep Compact Image Representation for Visual Tracking , 2013, NIPS.

[38]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[39]  Ruth Aylett,et al.  MY virtual graffiti system , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[40]  Toby P. Breckon,et al.  Real-Time Monocular Depth Estimation Using Synthetic Data with Domain Adaptation via Image Style Transfer , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  John Tran,et al.  cuDNN: Efficient Primitives for Deep Learning , 2014, ArXiv.

[42]  Paul G. Kry,et al.  Stippling with aerial robots , 2016, Expressive.

[43]  Gary W. Meyer,et al.  Automotive Spray Paint Simulation , 2008, ISVC.

[44]  Wookho Son,et al.  Virtual Reality based Paint Spray Training System , 2007, 2007 IEEE Virtual Reality Conference.

[45]  Oriol Vinyals,et al.  Synthesizing Programs for Images using Reinforced Adversarial Learning , 2018, ICML.

[46]  Sylvain Calinon,et al.  Learning dynamic graffiti strokes with a compliant robot , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[47]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Joseph A. Paradiso,et al.  Augmented Airbrush for Computer Aided Painting (CAP) , 2015, TOGS.

[49]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Tony F. Chan,et al.  High-Order Total Variation-Based Image Restoration , 2000, SIAM J. Sci. Comput..