Sketch-n-Sketch: Output-Directed Programming for SVG

For creative tasks, programmers face a choice: Use a GUI and sacrifice flexibility, or write code and sacrifice ergonomics? To obtain both flexibility and ease of use, a number of systems have explored a workflow that we call output-directed programming. In this paradigm, direct manipulation of the program's graphical output corresponds to writing code in a general-purpose programming language, and edits not possible with the mouse can still be enacted through ordinary text edits to the program. Such capabilities provide hope for integrating graphical user interfaces into what are currently text-centric programming environments. To further advance this vision, we present a variety of new output-directed techniques that extend the expressive power of Sketch-n-Sketch, an output-directed programming system for creating programs that generate vector graphics. To enable output-directed interaction at more stages of program construction, we expose intermediate execution products for manipulation and we present a mechanism for contextual drawing. Looking forward to output-directed programming beyond vector graphics, we also offer generic refactorings through the GUI, and our techniques employ a domain-agnostic provenance tracing scheme. To demonstrate the improved expressiveness, we implement a dozen new parametric designs in Sketch-n-Sketch without text-based edits. Among these is the first demonstration of building a recursive function in an output-directed programming setting.

[1]  Guy Pierra,et al.  The EBP System: Example Based Programming System for Parametric Design , 1996 .

[2]  Live end-user programming , 2016 .

[3]  Henry Lieberman,et al.  Watch what I do: programming by demonstration , 1993 .

[4]  Ravi Chugh,et al.  Semi-Automated SVG Programming via Direct Manipulation , 2016, UIST.

[5]  James Cheney,et al.  A Core Calculus for Provenance , 2012, POST.

[6]  Ben Shneiderman,et al.  Direct Manipulation: A Step Beyond Programming Languages , 1983, Computer.

[7]  Randall B. Smith,et al.  Directness and liveness in the morphic user interface construction environment , 1995, UIST '95.

[8]  Viktor Kuncak,et al.  Bidirectional evaluation with direct manipulation , 2018, Proc. ACM Program. Lang..

[9]  W. R. Sutherland,et al.  The on-line graphical specification of computer procedures , 1966 .

[10]  Armando Solar-Lezama,et al.  Learning to Infer Graphics Programs from Hand-Drawn Images , 2017, NeurIPS.

[11]  Nicholas Jackiw,et al.  The geometer's sketchpad: programming by geometry , 1993 .

[12]  Henry Lieberman,et al.  Mondrian: a teachable graphical editor , 1993, INTERCHI.

[13]  Mitchel Resnick,et al.  LogoBlocks: A Graphical Programming Language for Interacting with the World , 2000 .

[14]  Bogdan Dit,et al.  Integrated impact analysis for managing software changes , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[15]  Tovi Grossman,et al.  Object-Oriented Drawing , 2016, CHI.

[16]  Sumit Gulwani,et al.  Ringer: web automation by demonstration , 2016, OOPSLA.

[17]  Michael Gleicher,et al.  Drawing with constraints , 1994, The Visual Computer.

[18]  Sumit Gulwani,et al.  User Interaction Models for Disambiguation in Programming by Example , 2015, UIST.

[19]  Henry Lieberman Tinker: a programming by demonstration system for beginning programmers , 1993 .

[20]  David Kurlander Graphical editing by example (abstract) , 1993, CHI '93.

[21]  Ian H. Witten,et al.  Metamouse: specifying graphical procedures by example , 1989, SIGGRAPH.

[22]  Armando Solar-Lezama,et al.  Program synthesis by sketching , 2008 .

[23]  Rastislav Bodík,et al.  Rousillon: Scraping Distributed Hierarchical Web Data , 2018, UIST.

[24]  Ivan E. Sutherland,et al.  Sketchpad a Man-Machine Graphical Communication System , 1899, Outstanding Dissertations in the Computer Sciences.

[25]  Allan Heydon,et al.  The Juno-2 Constraint-Based Drawing Editor , 1994 .

[26]  Armando Solar-Lezama,et al.  DemoMatch: API discovery from demonstrations , 2017, PLDI.

[27]  Tao Xie,et al.  Automating presentation changes in dynamic web applications via collaborative hybrid analysis , 2012, SIGSOFT FSE.

[28]  Cormac Flanagan,et al.  Live Programming by Example : Using Direct Manipulation for Live Program Synthesis , 2016 .

[29]  Manu Sridharan,et al.  Refactoring with synthesis , 2013, OOPSLA.

[30]  Sumit Gulwani,et al.  QuickDraw: improving drawing experience for geometric diagrams , 2012, CHI.

[31]  Bogdan Dit,et al.  ImpactMiner: a tool for change impact analysis , 2014, ICSE Companion.

[32]  Radomír Mech,et al.  Supporting Expressive Procedural Art Creation through Direct Manipulation , 2017, CHI.

[33]  Ravi Chugh,et al.  Programmatic and direct manipulation, together at last , 2015, PLDI.

[34]  A. C. Hearn,et al.  REDUCE: a user-oriented interactive system for algebraic simplification , 1967, Symposium on Interactive Systems for Experimental Applied Mathematics.

[35]  Wilmot Li,et al.  Lillicon: using transient widgets to create scale variations of icons , 2015, ACM Trans. Graph..

[36]  Dean Jackson Scalable vector graphics (SVG): the world wide web consortium's recommendation for high quality web graphics , 2002, SIGGRAPH '02.

[37]  Alan Borning,et al.  The Programming Language Aspects of ThingLab, a Constraint-Oriented Simulation Laboratory , 1981, TOPL.

[38]  Alan Borning,et al.  Babelsberg/JS - A Browser-Based Implementation of an Object Constraint Language , 2014, ECOOP.

[39]  Anneli Folkesson,et al.  World Wide Web Consortium (W3C) , 2005 .

[40]  Steven K. Feiner,et al.  Graphical editing by example , 1993 .