Recognition System for On-Line Sketched Diagrams

We present our recent model of a diagram recognition engine. It extends our previous work which approaches the structural recognition as an optimization problem of choosing the best subset of symbol candidates. The main improvement is the integration of our own text separator into the pipeline to deal with text blocks occurring in diagrams. Second improvement is splitting the symbol candidates detection into two stages: uniform symbols detection and arrows detection. Text recognition is left for post processing when the diagram structure is already known. Training and testing of the engine was done on a freely available benchmark database of flowcharts. We correctly segmented and recognized 93.0% of the symbols having 55.1% of the diagrams recognized without any error. Considering correct stroke labeling, we achieved the precision of 95.7%. This result is superior to the state-of-the-art method with the precision of 92.4%. Additionally, we demonstrate the generality of the proposed method by adapting the system to finite automata domain and evaluating it on own database of such diagrams.

[1]  Harold Mouchère,et al.  First experiments on a new online handwritten flowchart database , 2011, Electronic Imaging.

[2]  Hidetoshi Miyao,et al.  On-Line Handwritten flowchart Recognition, Beautification and Editing System , 2012, 2012 International Conference on Frontiers in Handwriting Recognition.

[3]  Harold Mouchère,et al.  Interest of Syntactic Knowledge for On-Line Flowchart Recognition , 2011, GREC.

[4]  Václav Hlavác,et al.  Ten Lectures on Statistical and Structural Pattern Recognition , 2002, Computational Imaging and Vision.

[5]  Jianmin Zhao,et al.  Advances in Blended Learning: Second Workshop on Blended Learning, WBL 2008, Jinhua, China, Augustl 20-22, 2008. Revised Selected Papers , 2008 .

[6]  Louis Vuurpijl,et al.  Mode detection in on-line pen drawing and handwriting recognition , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[7]  Yuan Qi,et al.  Diagram structure recognition by Bayesian conditional random fields , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[8]  Liang Zhang,et al.  A Novel Pen-Based Flowchart Recognition System for Programming Teaching , 2008, WBL.

[9]  Paul A. Viola,et al.  Ambiguity and Constraint in Mathematical Expression Recognition , 1998, AAAI/IAAI.

[10]  Masakazu Suzuki,et al.  Mathematical formula recognition using virtual link network , 2001, Proceedings of Sixth International Conference on Document Analysis and Recognition.

[11]  Cheng-Lin Liu,et al.  Online Japanese Character Recognition Using Trajectory-Based Normalization and Direction Feature Extraction , 2006 .

[12]  Cheng-Lin Liu,et al.  Text/Non-text Ink Stroke Classification in Japanese Handwriting Based on Markov Random Fields , 2007 .

[13]  Václav Hlavác,et al.  Modeling Flowchart Structure Recognition as a Max-Sum Problem , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[14]  Raúl Rojas,et al.  Recognition of On-Line Handwritten Commutative Diagrams , 2009, 2009 10th International Conference on Document Analysis and Recognition.

[15]  Tomás Werner,et al.  A Linear Programming Approach to Max-Sum Problem: A Review , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Levent Burak Kara,et al.  Hierarchical parsing and recognition of hand-sketched diagrams , 2004, UIST '04.

[17]  Christian Viard-Gaudin,et al.  On-line hand-drawn electric circuit diagram recognition using 2D dynamic programming , 2009, Pattern Recognit..

[18]  Aurélie Lemaitre,et al.  Fusion of Statistical and Structural Information for Flowchart Recognition , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[19]  Stéphane Lavirotte,et al.  Mathematical formula recognition using graph grammar , 1998, Electronic Imaging.