Recognition of mathematics notation via computer using baseline structure

The spatial structure of mathematical expressions may be represented using the structure of baselines in an expression. This baseline structure may be represented as a tree, and then rewritten to represent di erent interpretations of spatial structure in an expression. This structure tree may also be translated into mathematical string languages such as LTEX, Maple etc. A parsing algorithm which extracts baseline structure in mathematics expressions from a list of attributed symbols is presented, along with an implementation of the algorithm. The implementation of the parsing algorithm has been interfaced with an online mathematics entry system. The resulting integrated system allowed testing of the parser on hundreds of input expressions, with encouraging results. Future work in the extension of the current mathematics recognition technique and application of direction-based recognition to other notations such as music notation and circuit diagrams are also discussed.

[1]  Dorothea Blostein.  General Diagram-Recognition Methodologies , 1995, GREC.

[2]  Stephane Lavirotte.  Optical formula recognition , 1997, Proceedings of the Fourth International Conference on Document Analysis and Recognition.

[3]  R. H. Anderson,et al.  Two-Dimensional Mathematical Notation , 1977 .

[4]  R. Furuta,et al.  TEX and METAFONT, new directions in typesetting , 1983, Proceedings of the IEEE.

[5]  Jean Paul Haton,et al.  A Syntactic Approach for Handwritten Mathematical Formula Recognition , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Horst Bunke.  Attributed Programmed Graph Grammars and Their Application to Schematic Diagram Interpretation , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Manfred K. Lang,et al.  A soft-decision approach for structural analysis of handwritten mathematical expressions , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[8]  Gerhard Rigoll,et al.  On-line handwritten formula recognition using statistical methods , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[9]  Andrea De Lucia,et al.  Towards efficient parsing of diagrammatic languages , 1994, AVI '94.

[10]  Dorothea Blostein,et al.  A Graph-Rewriting Paradigm for Discrete Relaxation: Application to Sheet-Music Recognition , 1998, Int. J. Pattern Recognit. Artif. Intell..

[11]  Nicholas J. Higham,et al.  Handbook of writing for the mathematical sciences , 1993 .

[12]  Richard J. Fateman,et al.  Progress in recognizing typeset mathematics , 1996, Electronic Imaging.

[13]  Stephan Baumann.  A simplified attributed graph grammar for high-level music recognition , 1995, Proceedings of 3rd International Conference on Document Analysis and Recognition.

[14]  M. Mahoney,et al.  History of Mathematics , 1924, Nature.

[15]  M. Lang,et al.  Symbol Segmentation and Recognition for Understanding Handwritten Mathematical Expressions , 1997 .

[16]  Tetsuya Sakurai,et al.  A methodology of parsing mathematical notation for mathematical computation , 1996, ISSAC '96.

[17]  P. A. Chou,et al.  Recognition of Equations Using a Two-Dimensional Stochastic Context-Free Grammar , 1989, Other Conferences.

[18]  C. Faure,et al.  Structural analysis of handwritten mathematical expressions , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.

[19]  William A. Martin,et al.  Computer input/output of mathematical expressions , 1971, SYMSAC '71.

[20]  Van Nam Tran,et al.  Syntactic pattern recognition , 1978 .

[21]  James Arvo,et al.  A Handwritting-Based Equation Editor , 1999, Graphics Interface.

[22]  Brad A. Myers,et al.  Taxonomies of visual programming and program visualization , 1990, J. Vis. Lang. Comput..

[23]  W. Cassie,et al.  Printing Mathematics , 2022, Nature.

[24]  Genny Tortora,et al.  A Parsing Methodology for the Implementation of Visual Systems , 1997, IEEE Trans. Software Eng..

[25]  Genny Tortora,et al.  A framework of syntactic models for the implementation of visual languages , 1997, Proceedings. 1997 IEEE Symposium on Visual Languages (Cat. No.97TB100180).

[26]  Giulia Pagallo.  Constrained Attribute Grammars for Recognition of Multi-dimensional Objects , 1998, SSPR/SPR.

[27]  Hsi-Jian Lee,et al.  Understanding mathematical expressions using procedure-oriented transformation , 1994, Pattern Recognit..

[28]  James R. Cordy,et al.  TXL: A Rapid Prototyping System for Programming Language Dialects , 1991, Comput. Lang..

[29]  Genny Tortora,et al.  Positional grammars: a formalism for LR-like parsing of visual languages , 1998 .

[30]  Yannis A. Dimitriadis,et al.  Towards an art based mathematical editor, that uses on-line handwritten symbol recognition , 1995, Pattern Recognit..

[31]  Gennaro Costagliola,et al.  Using Linear Positional Grammars for the LR Parsing of 2-D Symbolic Languages , 1999, Grammars.

[32]  Paul A. Viola,et al.  Ambiguity and Constraint in Mathematical Expression Recognition , 1998, AAAI/IAAI.

[33]  Dov Dori,et al.  The grammar of dimensions in machine drawings , 1988, Comput. Vis. Graph. Image Process..

[34]  Ch. Faure,et al.  AUTOMATIC PERCEPTION OF THE STRUCTURE OF HANDWRITTEN MATHEMATICAL EXPRESSIONS , 1990 .

[35]  T. S. El-Sheikh.  Recognition of handwritten Arabic mathematical formulas , 1990 .

[36]  Kim Marriott,et al.  A survey of visual language specification and recognition , 1998 .

[37]  Dorothea Blostein,et al.  Mathematics recognition using graph rewriting , 1995, Proceedings of 3rd International Conference on Document Analysis and Recognition.

[38]  T. L. H.,et al.  A History of Mathematical Notations , 1929, Nature.

[39]  Masayuki Okamoto,et al.  Structure analysis and recognition of mathematical expressions , 1995, Proceedings of 3rd International Conference on Document Analysis and Recognition.

[40]  Hsi-Jian Lee,et al.  Design of a mathematical expression recognition system , 1995, Proceedings of 3rd International Conference on Document Analysis and Recognition.

[41]  Shi-Kuo Chang,et al.  A method for the structural analysis of two-dimensional mathematical expressions , 1970, Inf. Sci..

[42]  Richard J. Fateman,et al.  Optical character recognition for typeset mathematics , 1994, ISSAC '94.

[43]  Masayuki Okamoto,et al.  An Experimental Implementation of a Document Recognition System for Papers Containing Mathematical Expressions , 1992 .