Understanding, Manipulating and Searching Hand-Drawn Concept Maps

Concept maps are an important tool to organize, represent, and share knowledge. Building a concept map involves creating text-based concepts and specifying their relationships with line-based links. Current concept map tools usually impose specific task structures for text and link construction, and may increase cognitive burden to generate and interact with concept maps. While pen-based devices (e.g., tablet PCs) offer users more freedom in drawing concept maps with a pen or stylus more naturally, the support for hand-drawn concept map creation and manipulation is still limited, largely due to the lack of methods to recognize the components and structures of hand-drawn concept maps. This article proposes a method to understand hand-drawn concept maps. Our algorithm can extract node blocks, or concept blocks, and link blocks of a hand-drawn concept map by combining dynamic programming and graph partitioning, recognize the text content of each concept node, and build a concept-map structure by relating concepts and links. We also design an algorithm for concept map retrieval based on hand-drawn queries. With our algorithms, we introduce structure-based intelligent manipulation techniques and ink-based retrieval techniques to support the management and modification of hand-drawn concept maps. Results from our evaluation study show high structure recognition accuracy in real time of our method, and good usability of intelligent manipulation and retrieval techniques.

[1]  G. Cox,et al.  ~ " " " ' l I ~ " " -" . : -· " J , 2006 .

[2]  C. V. Jawahar,et al.  Retrieval of online handwriting by synthesis and matching , 2009, Pattern Recognit..

[3]  Tsuhan Chen,et al.  Representations, feature extraction, matching and relevance feedback for sketch retrieval , 2003 .

[4]  Martin Szummer,et al.  Incorporating Context and User Feedback in Pen-Based Interfaces , 2004, AAAI Technical Report.

[5]  TianFeng,et al.  Understanding, Manipulating and Searching Hand-Drawn Concept Maps , 2011 .

[6]  Gang Wang,et al.  Multimodal error correction for continuous handwriting recognition in pen-based user interfaces , 2006, IUI '06.

[7]  Vladimir I. Levenshtein,et al.  Binary codes capable of correcting deletions, insertions, and reversals , 1965 .

[8]  Clifford Stein,et al.  Introduction to Algorithms, 2nd edition. , 2001 .

[9]  Dit-Yan Yeung,et al.  Mathematical expression recognition: a survey , 2000, International Journal on Document Analysis and Recognition.

[10]  Mark D. Gross,et al.  Stretch-A-Sketch: a dynamic diagrammer , 1994, Proceedings of 1994 IEEE Symposium on Visual Languages.

[11]  Hongan Wang,et al.  Crossmodal error correction of continuous handwriting recognition by speech , 2007, IUI '07.

[12]  Guozhong Dai,et al.  Structuralizing digital ink for efficient selection , 2006, IUI '06.

[13]  Thomas H. Cormen,et al.  Introduction to algorithms [2nd ed.] , 2001 .

[14]  Michelle X. Zhou,et al.  A graph-matching approach to dynamic media allocation in intelligent multimedia interfaces , 2005, IUI '05.

[15]  Kenneth D. Forbus,et al.  Sketching for knowledge capture: a progress report , 2002, IUI '02.

[16]  Beryl Plimmer,et al.  Intelligent mind-mapping , 2007, OZCHI '07.

[17]  Joseph D. Novak,et al.  Learning creating and using knowledge: Concept maps as facilitative tools , 1998 .

[18]  C A Nelson,et al.  Learning to Learn , 2017, Encyclopedia of Machine Learning and Data Mining.

[19]  Paul A. Viola,et al.  Recognition and grouping of handwritten text in diagrams and equations , 2004, Ninth International Workshop on Frontiers in Handwriting Recognition.

[20]  Xiangshi Ren,et al.  Structuralizing Freeform Notes by Implicit Sketch Understanding , 2002 .

[21]  Thomas P. Moran,et al.  Implicit structure for pen-based systems within a freeform interaction paradigm , 1995, CHI '95.

[22]  Shuang Liang,et al.  Sketch retrieval and relevance feedback with biased SVM classification , 2008, Pattern Recognit. Lett..

[23]  Sonia Halimi,et al.  THE CONCEPT MAP AS A COGNITIVE TOOL FOR SPECIALIZED INFORMATION RECALL , 2006 .

[24]  Hongan Wang,et al.  Structuring and manipulating hand-drawn concept maps , 2010, 2010 4th International Universal Communication Symposium.

[25]  Sigmar-Olaf Tergan,et al.  Digital Concept Maps for Managing Knowledge and Information , 2005, Knowledge and Information Visualization.

[26]  J. Novak.  The Theory Underlying Concept Maps and How To Construct Them , 2004 .

[27]  Anil K. Jain,et al.  Indexing and retrieval of on-line handwritten documents , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[28]  Stuart Russell,et al.  Statistical Visual Language Models for Ink Parsing , 2002 .

[29]  Mario Vento,et al.  An Improved Algorithm for Matching Large Graphs , 2001 .

[30]  Hongan Wang,et al.  Intelligent understanding of handwritten geometry theorem proving , 2010, IUI '10.

[31]  Levent Burak Kara,et al.  Hierarchical parsing and recognition of hand-sketched diagrams , 2007, SIGGRAPH '07.

[32]  Yang Li,et al.  Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes , 2007, UIST.

[33]  Levent Burak Kara,et al.  Combining geometry and domain knowledge to interpret hand-drawn diagrams , 2005, Comput. Graph..

[34]  Mario Vento,et al.  Thirty Years Of Graph Matching In Pattern Recognition , 2004, Int. J. Pattern Recognit. Artif. Intell..

[35]  Desney S. Tan,et al.  InkSeine: In Situ search for active note taking , 2007, CHI.

[36]  Randall Davis,et al.  Tahuti: a geometrical sketch recognition system for UML class diagrams , 2006, SIGGRAPH Courses.

[37]  Christine Alvarado,et al.  Multi-domain sketch understanding , 2007, SIGGRAPH Courses.

[38]  Levent Burak Kara,et al.  Hierarchical parsing and recognition of hand-sketched diagrams , 2004, UIST '04.

[39]  Randall Davis,et al.  Recognition of Hand Drawn Chemical Diagrams , 2007, AAAI.