Using Entropy to Distinguish Shape Versus Text in Hand-Drawn Diagrams

Most sketch recognition systems are accurate in recognizing either text or shape (graphic) ink strokes, but not both. Distinguishing between shape and text strokes is, therefore, a critical task in recognizing hand-drawn digital ink diagrams that contain text labels and annotations. We have found the 'entropy rate' to be an accurate criterion of classification. We found that the entropy rate is significantly higher for text strokes compared to shape strokes and can serve as a distinguishing factor between the two. Using a single feature -- zero-order entropy rate -- our system produced a correct classification rate of 92.06% on test data belonging to diagrammatic domain for which the threshold was trained on. It also performed favorably on an unseen domain for which no training examples were supplied.

[1]  Tracy Anne Hammond,et al.  LADDER: a language to describe drawing, display, and editing in sketch recognition , 2003, IJCAI.

[2]  Anil K. Jain,et al.  Structure in on-line documents , 2001, Proceedings of Sixth International Conference on Document Analysis and Recognition.

[3]  Randall Davis,et al.  Tahuti: a geometrical sketch recognition system for UML class diagrams , 2006, SIGGRAPH Courses.

[4]  Beryl Plimmer,et al.  Ink features for diagram recognition , 2007, SBIM '07.

[5]  Haym Hirsh,et al.  Using LSI for text classification in the presence of background text , 2001, CIKM '01.

[6]  Gerard Salton,et al.  Research and Development in Information Retrieval , 1982, Lecture Notes in Computer Science.

[7]  Christopher M. Bishop,et al.  Distinguishing text from graphics in on-line handwritten ink , 2004, Ninth International Workshop on Frontiers in Handwriting Recognition.

[8]  Yan Zhou,et al.  Enhancing Supervised Learning with Unlabeled Data , 2000, ICML.

[9]  Thomas F. Stahovich,et al.  SketchIT: a Sketch Interpretation Tool for Conceptual Mechanical Design , 1996 .

[10]  Christine J. Alvarado,et al.  A natural sketching environment : bringing the computer into early stages of mechanical design , 2000 .

[11]  Dean Rubine,et al.  Specifying gestures by example , 1991, SIGGRAPH.

[12]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[13]  Tracy Anne Hammond,et al.  SOUSA: sketch-based online user study applet , 2008, SBM'08.

[14]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[15]  Tracy Anne Hammond,et al.  GLADDER: Combining Gesture and Geometric Sketch Recognition , 2008, AAAI.

[16]  Ellen Yi-Luen Do,et al.  Ambiguous intentions: a paper-like interface for creative design , 1996, UIST '96.

[17]  Haym Hirsh,et al.  Integrating Background Knowledge into Nearest-Neighbor Text Classification , 2002, ECCBR.

[18]  Robert H. Sloan,et al.  Proceedings of the 15th Annual Conference on Computational Learning Theory , 2002 .

[19]  William W. Cohen.  Integration of heterogeneous databases without common domains using queries based on textual similarity , 1998, SIGMOD '98.

[20]  James A. Landay,et al.  Interactive sketching for the early stages of user interface design , 1995, CHI '95.

[21]  Adam Krzyżak,et al.  Methods of combining multiple classifiers and their applications to handwriting recognition , 1992, IEEE Trans. Syst. Man Cybern..

[22]  Joaquim A. Jorge,et al.  JavaSketchIt: Issues in Sketching the Look of User Interfaces , 2002 .

[23]  Haym Hirsh,et al.  Improving Short Text Classification Using Unlabeled Background Knowledge , 2000, ICML 2000.

[24]  Richard A. Harshman,et al.  Indexing by Latent Semantic Analysis , 1990, J. Am. Soc. Inf. Sci..

[25]  David Nahamoo,et al.  A Fast Statistical Mixture Algorithm for On-Line Handwriting Recognition , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Ayhan Demiriz,et al.  Semi-Supervised Support Vector Machines , 1998, NIPS.

[27]  Sebastian Thrun,et al.  Text Classification from Labeled and Unlabeled Documents using EM , 2000, Machine Learning.

[28]  Christine Alvarado,et al.  SketchREAD: a multi-domain sketch recognition engine , 2004, UIST '04.

[29]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[30]  Christian Heide Damm,et al.  Tool support for cooperative object-oriented design: gesture based modelling on an electronic whiteboard , 2000, CHI.

[31]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[32]  Levent Burak Kara,et al.  Combining geometry and domain knowledge to interpret hand-drawn diagrams , 2005, Comput. Graph..

[33]  William A. Gale,et al.  A sequential algorithm for training text classifiers , 1994, SIGIR '94.