Temporal sketch recognition in interspersed drawings

Sketch recognition has been recognized as an enabling technology for pen-based interfaces. Previous work in the field has shown that in certain domains the stroke orderings used when drawing objects contain temporal patterns that can aid recognition. So far, systems that use temporal information for recognition have assumed that objects are drawn one at a time. This paper shows how this assumption can be relaxed to permit temporal interspersing of strokes from different objects. We describe a statistical framework based on Dynamic Bayesian Networks that explicitly models the fact that objects can be drawn interspersed. We present recognition results for hand-drawn electronic circuit diagrams. The results show that handling interspersed drawing provides a significant increase in accuracy.

[1]  David Heckerman,et al.  Knowledge Representation and Inference in Similarity Networks and Bayesian Multinets , 1996, Artif. Intell..

[2]  Michael C. Horsch,et al.  Dynamic Bayesian networks , 1990 .

[3]  Levent Burak Kara,et al.  Combining geometry and domain knowledge to interpret hand-drawn diagrams , 2005, Comput. Graph..

[4]  Geoffrey Zweig,et al.  The graphical models toolkit: An open source software system for speech and time-series processing , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[5]  B. Tversky,et al.  What Does Drawing Reveal about Thinking? , 1999 .

[6]  Randall Davis,et al.  Sketch Interpretation Using Multiscale Models of Temporal Patterns , 2007, IEEE Computer Graphics and Applications.

[7]  Kevin Murphy,et al.  Dynamic Bayesian Networks , 2002 .

[8]  Christine Alvarado,et al.  SketchREAD: a multi-domain sketch recognition engine , 2004, UIST '04.

[9]  Randall Davis,et al.  Automatically Transforming Symbolic Shape Descriptions for Use in Sketch Recognition , 2004, AAAI.

[10]  Thomas F. Stahovich,et al.  Sketch based interfaces: early processing for sketch understanding , 2001, PUI '01.

[11]  Jeff A. Bilmes,et al.  Dynamic Bayesian Multinets , 2000, UAI.

[12]  James V. Mahoney,et al.  Three main concerns in sketch recognition and an approach to addressing them , 2002 .

[13]  Balaji Krishnapuram,et al.  Generative models and Bayesian model comparison for shape recognition , 2004, Ninth International Workshop on Frontiers in Handwriting Recognition.

[14]  Marjorie Skubic,et al.  Hidden Markov Model Symbol Recognition for Sketch-Based Interfaces , 2004, AAAI Technical Report.

[15]  Randall Davis,et al.  HMM-based efficient sketch recognition , 2005, IUI.

[16]  Craig Boutilier,et al.  Context-Specific Independence in Bayesian Networks , 1996, UAI.

[17]  Martin Szummer,et al.  A Graphical Model for Simultaneous Partitioning and Labeling , 2005, AISTATS.

[18]  Paul A. Viola,et al.  Recognition and grouping of handwritten text in diagrams and equations , 2004, Ninth International Workshop on Frontiers in Handwriting Recognition.

[19]  Gregory Dudek,et al.  Sketch Interpretation and Refinement Using Statistical Models , 2004, Rendering Techniques.

[20]  C. Alvarado,et al.  Properties of Real-World Digital Logic Diagrams , 2007, First International Workshop on Pen-Based Learning Technologies (PLT 2007).