A Bayesian Approach to Sketch Recognition with Auto-Completion

Sketching is a natural form of human interaction that is employed in a variety of areas, such as engineering drawings or classroom teaching. Recognition of hand drawn sketches is a challenging problem due to the variability in hand drawing, variability in the drawing order of strokes, and the similarity of sketch classes. In this work, we present a system that classifies sketches as they are being drawn, in order to increase sketching throughput. In cases where the label of the partial sketch can not be predicted confidently, our system delays the classification decision until more information becomes available. The system consists of a semi-supervised clustering that clusters similar sketches followed by supervised classification that deals with the identification of the instances in a cluster. This setup allows us to deal with the ambiguity involved in classifying partial sketches that may belong to more than one category. We report experiments using the COAD and NicIcon databases, showing close to perfect classification results for full and partial sketches.

[1]  Louis Vuurpijl,et al.  The NicIcon database of handwritten icons , 2008, ICFHR 2008.

[2]  T. Metin Sezgin,et al.  Feature Extraction and Classifier Combination for Image-based Sketch Recognition , 2010, SBIM.

[3]  Bart Lamiroy,et al.  An Incremental On-line Parsing Algorithm for Recognizing Sketching Diagrams , 2007, Ninth International Conference on Document Analysis and Recognition (ICDAR 2007).

[4]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[5]  Gregory Dudek,et al.  Sketch Interpretation and Refinement Using Statistical Models , 2004, Rendering Techniques.

[6]  Peter van Sommers,et al.  Drawing and Cognition: Descriptive and Experimental Studies of Graphic Production Processes , 1984 .

[7]  Randall Davis,et al.  Sketch recognition in interspersed drawings using time-based graphical models , 2008, Comput. Graph..

[8]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[9]  James Arvo,et al.  Fluid sketches: continuous recognition and morphing of simple hand-drawn shapes , 2000, UIST '00.

[10]  Peter Tandler,et al.  Using Incremental Gesture Recognition to Provide Immediate Feedback while Drawing Pen Gestures , 2001 .

[11]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[12]  Peter Robinson,et al.  A Multimodal Interface for Road Design , 2009 .

[13]  Randall Davis,et al.  A Visual Approach to Sketched Symbol Recognition , 2009, IJCAI.

[14]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[15]  Claire Cardie,et al.  Proceedings of the Eighteenth International Conference on Machine Learning, 2001, p. 577–584. Constrained K-means Clustering with Background Knowledge , 2022 .

[16]  Guozhong Dai,et al.  Sketch recognition with continuous feedback based on incremental intention extraction , 2005, IUI.