LADDER, a sketching language for user interface developers

Sketch recognition systems are currently being developed for many domains, but can be time consuming to build if they are to handle the intricacies of each domain. In order to aid sketch-based user interface developers, we have developed tools to simplify the development of a new sketch recognition interface. We created LADDER, a language to describe how sketched diagrams in a domain are drawn, displayed, and edited. We then automatically transform LADDER structural descriptions into domain specific shape recognizers, editing recognizers, and shape exhibitors for use in conjunction with a domain independent sketch recognition system, creating a sketch recognition system for that domain. We have tested our framework by writing several domain descriptions and automatically generating a domain specific sketch recognition system from each description.

[1]  George Stiny,et al.  Shape Grammars and the Generative Specification of Painting and Sculpture , 1971, IFIP Congress.

[2]  Robert L. Mason,et al.  Some practical aspects of covariance estimation , 1985, Pattern Recognit. Lett..

[3]  N. D. Belajcic.  Computer Implementation of Shape Grammars , 1988 .

[4]  Dean Rubine,et al.  Specifying gestures by example , 1991, SIGGRAPH.

[5]  Efficient analysis of complex diagrams using constraint-based parsing , 1995, Proceedings of 3rd International Conference on Document Analysis and Recognition.

[6]  James A. Landay,et al.  Interactive sketching for the early stages of user interface design , 1995, CHI '95.

[7]  Genny Tortora,et al.  Automatic Generation of Visual Programming Environments , 1995, Computer.

[8]  Thomas F. Stahovich,et al.  SketchIT: a Sketch Interpretation Tool for Conceptual Mechanical Design , 1996 .

[9]  Ellen Yi-Luen Do,et al.  Demonstrating the electronic cocktail napkin: a paper-like interface for early design , 1996, CHI Conference Companion.

[10]  E. J. Friedman-hill,et al.  Jess, the Java expert system shell , 1997 .

[11]  Eric Lecolinet.  Designing GUIs by sketch drawing and visual programming , 1998, AVI '98.

[12]  James Gips,et al.  Computer Implementation of Shape Grammars , 1999 .

[13]  Robert J. K. Jacob,et al.  A software model and specification language for non-WIMP user interfaces , 1999, TCHI.

[14]  Victor Zue,et al.  Conversational interfaces: advances and challenges , 1997, Proceedings of the IEEE.

[15]  Christian Heide Damm,et al.  Tool support for cooperative object-oriented design: gesture based modelling on an electronic whiteboard , 2000, CHI.

[16]  Z E. Jess Friedman-hill,et al.  The Java Expert System Shell , 2000 .

[17]  Oliver Bimber,et al.  A multi-layered architecture for sketch-based interaction within virtual environments , 2000, Comput. Graph..

[18]  Christine J. Alvarado,et al.  A natural sketching environment : bringing the computer into early stages of mechanical design , 2000 .

[19]  Mark W. Newman,et al.  DENIM: finding a tighter fit between tools and practice for Web site design , 2000, CHI.

[20]  W. Godwin.  Article in Press , 2000 .

[21]  Sean Jy-Shyang Chen,et al.  An interactive system for recognizing hand drawn UML diagrams , 2000, CASCON.

[22]  Ellen Yi-Luen Do.  VR Sketchpad. Create Instant 3D Worlds by Sketching on a Transparent Window , 2001 .

[23]  James V. Mahoney,et al.  Handling ambiguity in constraint-based recognition of stick figure sketches , 2001, IS&T/SPIE Electronic Imaging.

[24]  James A. Landay,et al.  Quill: a gesture design tool for pen-based user interfaces , 2001 .

[25]  R. Davis,et al.  Perceptually Based Learning of Shape Descriptions , 2002 .

[26]  L. Kara,et al.  Recognizing Multi-Stroke Symbols , 2002 .

[27]  Joaquim A. Jorge,et al.  JavaSketchIt: Issues in Sketching the Look of User Interfaces , 2002 .

[28]  Stuart Russell,et al.  Statistical Visual Language Models for Ink Parsing , 2002 .

[29]  James V. Mahoney,et al.  Three main concerns in sketch recognition and an approach to addressing them , 2002 .

[30]  Tracy Anne Hammond,et al.  LADDER: a language to describe drawing, display, and editing in sketch recognition , 2003, IJCAI.

[31]  Christine Alvarado,et al.  SketchREAD: a multi-domain sketch recognition engine , 2004, UIST '04.

[32]  Randall Davis,et al.  Automatically Transforming Symbolic Shape Descriptions for Use in Sketch Recognition , 2004, AAAI.

[33]  Randall Davis,et al.  SHADY: A Shape Description Debugger for Use in Sketch Recognition , 2004, AAAI Technical Report.

[34]  Randall Davis,et al.  Tahuti: a geometrical sketch recognition system for UML class diagrams , 2006, SIGGRAPH Courses.

[35]  Manuel J. Fonseca,et al.  Sketching User Interfaces with Visual Patterns , .