Games for sketch data collection

This article describes sketching games made for the purpose of collecting data about how people make and describe hand-made drawings. The approach leverages human computation, whereby players provide information about drawings in exchange for entertainment. The games facilitate the collection of raw sketch input and associates it with human-provided text descriptions. Researchers may browse and download this data for their own purposes such as training sketch recognizers. Two systems with distinct game mechanics are described: Picture-phone and Stellasketch. The system architectures are briefly presented, followed by a discussion of our initial results using sketching games as a research platform for sketch recognition and interaction.

[1]  Mark W. Newman,et al.  Informal PUIs: No Recognition Required , 2002 .

[2]  Randall Davis,et al.  Perceptually based learning of shape descriptions for sketch recognition , 2004, AAAI.

[3]  R. Davis,et al.  Perceptually Based Learning of Shape Descriptions , 2002 .

[4]  Tracy Anne Hammond,et al.  SOUSA: sketch-based online user study applet , 2008, SBM'08.

[5]  Ellen Yi-Luen Do,et al.  Design sketches and sketch design tools , 2005, Knowl. Based Syst..

[6]  Levent Burak Kara,et al.  An image-based, trainable symbol recognizer for hand-drawn sketches , 2005, Comput. Graph..

[7]  Randall Davis,et al.  Interactive learning of structural shape descriptions from automatically generated near-miss examples , 2006, IUI '06.

[8]  Robert F. Sproull,et al.  Principles in interactive computer graphics , 1973 .

[9]  Timothy Chklovski,et al.  Designing interfaces for guided collection of knowledge about everyday objects from volunteers , 2005, IUI.

[10]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[11]  G. Griffin,et al.  Caltech-256 Object Category Dataset , 2007 .

[12]  Norio Akamatsu,et al.  Geometric Shape Recognition with Fuzzy Filtered Input to a Backpropagation Neural Network , 1995, IEICE Trans. Inf. Syst..

[13]  Christine Alvarado,et al.  Dynamically constructed Bayes nets for multi-domain sketch understanding , 2005, IJCAI.

[14]  Christine Alvarado,et al.  ETCHA Sketches: Lessons Learned from Collecting Sketch Data , 2004, AAAI Technical Report.

[15]  Yong Wang,et al.  A data collection tool for sketched diagrams , 2008, SBM'08.

[16]  Rich Gossweiler,et al.  WWW 2009 MADRID! Track: User Interfaces and Mobile Web / Session: User Interfaces What’s Up CAPTCHA? A CAPTCHA Based on Image Orientation , 2022 .

[17]  Joaquim A. Jorge,et al.  CALI: An Online Scribble Recognizer for Calligraphic Interfaces , 2002 .

[18]  Randall Davis,et al.  LADDER, a sketching language for user interface developers , 2005, Comput. Graph..

[19]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[20]  Dean Rubine,et al.  Specifying gestures by example , 1991, SIGGRAPH.

[21]  Jason Hong,et al.  Computational Support for Sketching in Design: A Review , 2009, Found. Trends Hum. Comput. Interact..

[22]  Henry Lieberman,et al.  AnalogySpace: Reducing the Dimensionality of Common Sense Knowledge , 2008, AAAI.

[23]  Gabe Johnson,et al.  Picturephone: A Game for Sketch Data Capture , 2009 .

[24]  Ellen Yi-Luen Do,et al.  Ambiguous intentions: a paper-like interface for creative design , 1996, UIST '96.

[25]  Bernd Neumann,et al.  Adaptable drawing interpretation using object-oriented and constraint-based graphic specification , 1993, Proceedings of 2nd International Conference on Document Analysis and Recognition (ICDAR '93).

[26]  Gennaro Costagliola,et al.  Towards Semantic Role Labeling of Hand-drawn Sketches , 2008 .

[27]  Christine Alvarado,et al.  A Framework for Multi-Domain Sketch Recognition , 2002 .

[28]  Randall Davis,et al.  LADDER, a sketching language for user interface developers , 2005, Comput. Graph..

[29]  Randall Davis,et al.  Sketch Interpretation Using Multiscale Models of Temporal Patterns , 2007, IEEE Computer Graphics and Applications.

[30]  Yang Li,et al.  Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes , 2007, UIST.