Real-time drawing assistance through crowdsourcing

We propose a new method for the large-scale collection and analysis of drawings by using a mobile game specifically designed to collect such data. Analyzing this crowdsourced drawing database, we build a spatially varying model of artistic consensus at the stroke level. We then present a surprisingly simple stroke-correction method which uses our artistic consensus model to improve strokes in real-time. Importantly, our auto-corrections run interactively and appear nearly invisible to the user while seamlessly preserving artistic intent. Closing the loop, the game itself serves as a platform for large-scale evaluation of the effectiveness of our stroke correction algorithm.

[1]  Darren Leigh,et al.  Under the table interaction , 2006, UIST.

[2]  Daniel Dixon,et al.  iCanDraw: using sketch recognition and corrective feedback to assist a user in drawing human faces , 2010, CHI.

[3]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[4]  Patrick Baudisch,et al.  Precise selection techniques for multi-touch screens , 2006, CHI.

[5]  Yong Jae Lee,et al.  ShadowDraw: real-time user guidance for freehand drawing , 2011, ACM Trans. Graph..

[6]  Eitan Grinspun,et al.  Diamonds From the Rough: Improving Drawing, Painting, and Singing via Crowdsourcing , 2012, HCOMP@AAAI.

[7]  Daniel Vogel,et al.  Shift: a technique for operating pen-based interfaces using touch , 2007, CHI.

[8]  Daniel Vogel,et al.  HybridPointing: fluid switching between absolute and relative pointing with a direct input device , 2006, UIST.

[9]  Levent Burak Kara,et al.  Beautification of Design Sketches Using Trainable Stroke Clustering and Curve Fitting , 2011, IEEE Transactions on Visualization and Computer Graphics.

[10]  Ravin Balakrishnan,et al.  Elasticurves: exploiting stroke dynamics and inertia for the real-time neatening of sketched 2D curves , 2011, UIST '11.

[11]  Daniel J. Wigdor,et al.  Ripples: utilizing per-contact visualizations to improve user interaction with touch displays , 2009, UIST '09.

[12]  Patrick Baudisch,et al.  Lucid touch: a see-through mobile device , 2007, UIST.

[13]  C. Lawrence Zitnick,et al.  Handwriting beautification using token means , 2013, ACM Trans. Graph..

[14]  Olga Sorkine-Hornung,et al.  On Linear Variational Surface Deformation Methods , 2008, IEEE Transactions on Visualization and Computer Graphics.

[15]  Dorin Comaniciu,et al.  Mean shift analysis and applications , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[16]  Marc Alexa,et al.  How do humans sketch objects? , 2012, ACM Trans. Graph..

[17]  Xiaojun Bi,et al.  RearType: text entry using keys on the back of a device , 2010, Mobile HCI.

[18]  William A. Barrett,et al.  Intelligent scissors for image composition , 1995, SIGGRAPH.

[19]  Manuel Blum,et al.  Peekaboom: a game for locating objects in images , 2006, CHI.

[20]  Shumin Zhai,et al.  High precision touch screen interaction , 2003, CHI '03.

[21]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[22]  Daniel J. Wigdor,et al.  Imprecision, Inaccuracy, and Frustration: The Tale of Touch Input , 2010, Tabletops.

[23]  Adam Finkelstein,et al.  Where do people draw lines? , 2008, ACM Trans. Graph..