An efficient graph-based symbol recognizer

We describe a trainable symbol recognizer for pen-based user interfaces. Symbols are represented internally as attributed relational graphs that describe both the geometry and topology of the symbols. Symbol recognition reduces to the task of finding the definition symbol whose attributed relational graph best matches that of the unknown symbol. One challenge addressed in the current work is how to perform this graph matching in an effi- cient fashion so as to achieve interactive performance. We present four approximate graph matching techniques: Stochastic Matching, which is based on stochastic search; Error-driven Matching, which uses local matching errors to drive the solution to an optimal match; Greedy Matching, which uses greedy search; and Sort Matching, which relies on geometric information to accelerate the matching. Finally, we present promising results of initial user studies, and discuss the tradeoffs between the various matching techniques.

[1]  Thomas F. Stahovich Segmentation of Pen Strokes Using Pen Speed , 2004, AAAI Technical Report.

[2]  Julian R. Ullmann,et al.  An Algorithm for Subgraph Isomorphism , 1976, J. ACM.

[3]  Ernest Valveny,et al.  Symbol Recognition: Current Advances and Perspectives , 2001, GREC.

[4]  Randall Davis,et al.  Automatically Transforming Symbolic Shape Descriptions for Use in Sketch Recognition , 2004, AAAI.

[5]  A. Richard Newton,et al.  Sketched symbol recognition using Zernike moments , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[6]  Seong-Whan Lee,et al.  Recognizing Hand-Drawn Electrical Circuit Symbols with Attributed Graph Matching , 1992 .

[7]  Levent Burak Kara,et al.  Hierarchical parsing and recognition of hand-sketched diagrams , 2004, UIST '04.

[8]  Dean Rubine,et al.  Specifying gestures by example , 1991, SIGGRAPH.

[9]  Nicholas E. Matsakis Recognition of Handwritten Mathematical Expressions , 1999 .

[10]  Mark D. Gross,et al.  Recognizing and interpreting diagrams in design , 1994, AVI '94.

[11]  Joaquim A. Jorge,et al.  Cascading recognizers for ambiguous calligraphic interaction , 2004, SBM'04.

[12]  Zhaohui Huang,et al.  Affine-invariant B-spline moments for curve matching , 1996, IEEE Trans. Image Process..

[13]  Sven J. Dickinson,et al.  Introduction to the Special Section on Graph Algorithms in Computer Vision , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Levent Burak Kara,et al.  An image-based, trainable symbol recognizer for hand-drawn sketches , 2005, Comput. Graph..

[15]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[16]  L. Kara,et al.  Recognizing Multi-Stroke Symbols , 2002 .

[17]  Mario Vento,et al.  Thirty Years Of Graph Matching In Pattern Recognition , 2004, Int. J. Pattern Recognit. Artif. Intell..

[18]  A. Newton,et al.  Sketched symbol recognition using Zernike moments , 2004, ICPR 2004.

[19]  Randall Davis,et al.  HMM-based efficient sketch recognition , 2005, IUI.

[20]  Levent Burak Kara,et al.  Hierarchical parsing and recognition of hand-sketched diagrams , 2007, SIGGRAPH '07.

[21]  Levent Burak Kara,et al.  Combining geometry and domain knowledge to interpret hand-drawn diagrams , 2005, Comput. Graph..

[22]  Joaquim A. Jorge,et al.  CALI: An Online Scribble Recognizer for Calligraphic Interfaces , 2002 .

[23]  Frank Van Reeth,et al.  Improving Sketching by Utilizing Haptic Feedback , 2002 .