Portrait drawing by Paul the robot

We describe Paul, a robotic installation that produces observational face drawings of people. Paul is a naive drawer: it does not have highlevel knowledge of the structures constitutive of the human face (such as the mouth, nose, eyes) nor the capability of learning expertise based on experience as a human would. However, Paul is able to draw using the equivalent of an artist's stylistic signature based on a number of processes mimicking drawing skills and technique, which together form a drawing cycle. Furthermore, we present here our first efforts in implementing two different versions of visual feedback to permit the robot to iteratively augment and improve a drawing which initially is built from a process of salient lines recovery. The first form of visual feedback we study we refer to as computational as it involves a purely internal (memory-based) representation of regions to render via shading by the robot. The second version we call physical as it involves the use of a camera as an 'eye' taking new snapshots of the artefact in progress. This is then analysed to take decisions on where and how to render shading next. A main point we emphasise in this work is the issue of embodiment of graphical systems, in our case in a robotic platform. We present our arguments in favour of such a position for the graphics community to reflect upon. Finally, we emphasise that the drawings produced by Paul have been considered of interest by fine art professionals in recent international art fairs and exhibitions, as well as by the public at large. One drawing is now in the Victoria and Albert museum collection. We identify a number of factors that may account for such perceived qualities of the produced drawings. Graphical abstractPaul drawing at the Merge Festival Gallery, London, October 2012. Paul is a robotic installation that produces observational face drawings of people using the equivalent of an artist s stylistic signature forming a drawing cycle with visual feedback.Display Omitted HighlightsÂ? Paul is a robotic system which integrates graphics with vision for face portraying. Â? The NPR community can explore the embodiment of artistic skills jointly with software. Â? We design, implement and test computational (software only) visual feedback. Â? We also design, implement and test physical (hardware and software) visual feedback. Â? The production of Paul is being exhibited internationally in well known art venues.

[1]  J. Daugman Two-dimensional spectral analysis of cortical receptive field profiles , 1980, Vision Research.

[2]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[3]  Jasia Reichardt Machines and Art , 1987 .

[4]  P. Sinha,et al.  Face Recognition by Humans: , 2005 .

[5]  Cynthia Breazeal,et al.  Designing sociable robots , 2002 .

[6]  Aaron Hertzmann,et al.  Non-Photorealistic Rendering and the science of art , 2010, NPAR.

[7]  Karl F. MacDorman,et al.  Androids as an Experimental Apparatus: Why Is There an Uncanny Valley and Can We Exploit It? , 2005 .

[8]  David Salesin,et al.  Image-Based Hair Capture by Inverse Lighting , 2002, Graphics Interface.

[9]  Nicolai Petkov,et al.  Computational models of visual neurons specialised in the detection of periodic and aperiodic oriented visual stimuli: bar and grating cells , 1997, Biological Cybernetics.

[10]  Susan E. Brennan,et al.  From the Leonardo Archive , 2007, Leonardo.

[11]  Aude Billard,et al.  A humanoid robot drawing human portraits , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[12]  Michael Leyton The Structure of Paintings , 2006 .

[13]  Tobias Isenberg,et al.  Automatically Mimicking Unique Hand-Drawn Pencil Lines , 2009, Comput. Graph..

[14]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[15]  J. P. Jones,et al.  An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[16]  Hans Richter,et al.  Dada: art and anti-art , 1967 .

[17]  Semir Zeki,et al.  Art and the brain , 1998 .

[18]  Aaron Hertzmann,et al.  Painterly rendering with curved brush strokes of multiple sizes , 1998, SIGGRAPH.

[19]  John Collomosse,et al.  Painterly rendering using image salience , 2002, Proceedings 20th Eurographics UK Conference.

[20]  E. Kac Foundation and Development of Robotic Art , 1997 .

[21]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[22]  R. Malina Aaron’s Code: Meta-Art, Artificial Intelligence and the Work of Harold Cohen by Pamela McCorduck (review) , 2017 .

[23]  D. Cohen,et al.  Why can't most people draw what they see? , 1997, Journal of experimental psychology. Human perception and performance.

[24]  R. L. Solso The Psychology of Art and the Evolution of the Conscious Brain , 2003 .

[25]  Leonel Moura Artist A New Kind of Art : The Robotic Action Painter , 2008 .

[26]  Henrik Schiøler,et al.  Sociable Robots Through Self-Maintained Energy , 2006 .

[27]  Paul Haeberli,et al.  Paint by numbers: abstract image representations , 1990, SIGGRAPH.

[28]  M. Leyton Symmetry, Causality, Mind , 1999 .

[29]  Aleksandar Zivanovic,et al.  The development of a cybernetic sculptor: Edward Ihnatowicz and the senster , 2005, C&C '05.

[30]  Deanna Petherbridge The Primacy of Drawing: Histories and Theories of Practice , 1991 .

[31]  O. Schwartz,et al.  Visuomotor characterization of eye movements in a drawing task , 2009, Vision Research.

[32]  Harold Cohen,et al.  How to Draw Three People in a Botanical Garden , 1988, AAAI.

[33]  V. Gallese,et al.  Motion, emotion and empathy in esthetic experience , 2007, Trends in Cognitive Sciences.

[34]  Oliver Deussen,et al.  Feedback-guided stroke placement for a painting machine , 2012, CAe '12.

[35]  Pawan Sinha,et al.  Face Recognition by Humans: Nineteen Results All Computer Vision Researchers Should Know About , 2006, Proceedings of the IEEE.

[36]  Bruce Gooch,et al.  Non-photorealistic rendering , 2001 .

[37]  Frederic Fol Leymarie,et al.  Sketches by Paul the robot , 2012, CAe '12.

[38]  Fumiya Iida,et al.  The challenges ahead for bio-inspired 'soft' robotics , 2012, CACM.

[39]  Petra Gemeinboeck,et al.  Zwischenräume : The Machine as Voyeur , 2010 .

[40]  Giorgio Metta,et al.  YARP: Yet Another Robot Platform , 2006 .

[41]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[42]  C. N. Thai,et al.  Using Robotis Bioloid systems for instructional Robotics , 2011, 2011 Proceedings of IEEE Southeastcon.

[43]  Frederic Fol Leymarie,et al.  Simulating the Grassfire Transform Using an Active Contour Model , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Paul Granjon Performing with machines and machines that perform , 2008 .

[45]  Benjamin B. Kimia,et al.  From the Infinitely Large to the Infinitely Small , 2008, Medial Representations.

[46]  Yeung Yam,et al.  Preliminary study on vision-based pen-and-ink drawing by a robotic manipulator , 2009, 2009 IEEE/ASME International Conference on Advanced Intelligent Mechatronics.

[47]  P. Burt Fast filter transform for image processing , 1981 .

[48]  A. Pignocchi,et al.  How the intentions of the draftsman shape perception of a drawing , 2010, Consciousness and Cognition.

[49]  Patrick Tresset,et al.  Skediomata: Guinea Pig and Performer , 2011 .

[50]  Edward H. Adelson,et al.  PYRAMID METHODS IN IMAGE PROCESSING. , 1984 .

[51]  Nikolay Petkov,et al.  Biologically motivated computationally intensive approaches to image pattern recognition , 1995, Future Gener. Comput. Syst..

[52]  Massimo Bergamasco,et al.  Robotic creatures: Anthropomorphism and interaction in contemporary art , 2010, 19th International Symposium in Robot and Human Interactive Communication.

[53]  Ichiro Fujinaga,et al.  Gamera: A Python-based Toolkit for Structured Document Recognition , 2001 .

[54]  M. Babcock,et al.  Perception of dynamic information in static handwritten forms. , 1988, The American journal of psychology.

[55]  Patrick A. Tresset,et al.  Generative Portrait Sketching , 2005 .