Using Diagram Generation Software to Improve Diagram Recognition: A Case Study of Music Notation

Diagrams are widely used in society to transmit information such as circuit designs, music, mathematical formulae, architectural plans, and molecular structure. Computers must process diagrams both as images (marks on paper) and as information. A diagram recognizer translates from image to information and a diagram generator translates from information to image. Current technology for diagram generation is ahead of the technology for diagram recognition. Diagram generators have extensive knowledge of notational conventions which relate to readability and aesthetics, whereas current diagram recognizers focus on the hard constraints of the notation. To create a recognizer capable of exploiting layout information, it is expedient to reuse the expertise in existing diagram generators. In particular, we discuss the use of Lime (our editor and generator for music notation) to proofread and correct the raw output of MIDIScan (a third-party commercial recognizer for music notation). Over the past several years, this combination of software has been distributed to thousands of users.

[1]  Matti Pietikäinen,et al.  Benchmarking system for document analysis algorithms , 1998, Electronic Imaging.

[2]  Alan Belkin.  Macintosh Notation Software: Present and Future , 1994 .

[3]  Edward R. Tufte,et al.  The Visual Display of Quantitative Information , 1986 .

[4]  Karl Tombre,et al.  Graphics recognition : algorithms and systems : Second International Workshop, GREC ʾ97, Nancy, France, August 22-23, 1997 : selected papers , 1998 .

[5]  David A. Maltz,et al.  Markov source model for printed music decoding , 1996, J. Electronic Imaging.

[6]  T. W. Chaundy,et al.  The Printing of Mathematics , 1954 .

[7]  Luigi P. Cordella,et al.  An Alternative Approach to the Performance Evaluation of Thinning Algorithms for Document Processing Applications , 1995, GREC.

[8]  John J. Bertin,et al.  The semiology of graphics , 1983 .

[9]  David Bainbridge,et al.  Extensible optical music recognition , 1997 .

[10]  Lawrence O'Gorman,et al.  Document Image Analysis Systems - Guest Editors' Introduction to the Special Issue , 1992, Computer.

[11]  Robert M. Haralick,et al.  Document image understanding: geometric and logical layout , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Gardner Read,et al.  Music notation;: A manual of modern practice , 1972 .

[13]  J. Artur Serrano,et al.  The use of semantic constraints on diagram editors , 1995, Proceedings of Symposium on Visual Languages.

[14]  Dov Dori,et al.  The representation of document structure: a generic object-process analysis , 1995 .

[15]  Dov Dori,et al.  A Proposed Scheme for Performance Evaluation of Graphics/Text Separation Algorithms , 1997, GREC.

[16]  Tomek Strzalkowski,et al.  Reversible logic grammars for natural language parsing and generation , 1990, Comput. Intell..

[17]  Dorothea Blostein.  General Diagram-Recognition Methodologies , 1995, GREC.

[18]  Dorothea Blostein,et al.  RECOGNITION OF MATHEMATICAL NOTATION , 1997 .

[19]  David Bainbridge,et al.  AUTOMATIC READING OF MUSIC NOTATION , 1997 .

[20]  Robert M. Haralick,et al.  A Performance Evaluation Protocol for Graphics Recognition Systems , 1997, GREC.

[21]  Walter F. Tichy,et al.  Edge: An extendible graph editor , 1990, Softw. Pract. Exp..

[22]  T. W. Chaundy,et al.  Rules for Compositors and Readers at the University Press, Oxford , 1928, Nature.

[23]  Lippold Haken,et al.  The lime music editor: A diagram editor involving complex translations , 1994, Softw. Pract. Exp..

[24]  Arnold W. M. Smeulders,et al.  Design and Performance in Object Recognition , 1997, GREC.

[25]  Carlo Batini,et al.  Automatic graph drawing and readability of diagrams , 1988, IEEE Trans. Syst. Man Cybern..

[26]  Bülent Sankur,et al.  The performance evaluation of thresholding algorithms for optical character recognition , 1997, Proceedings of the Fourth International Conference on Document Analysis and Recognition.

[27]  Paul G. Sorenson,et al.  Towards the Automatic Generation of Software Diagrams , 1991, IEEE Trans. Software Eng..

[28]  Atul K. Chhabra,et al.  Symbol Recognition : An Overview , 2005 .

[29]  Lippold Haken,et al.  Justification of printed music , 1991, CACM.

[30]  Neff Walker,et al.  A classification of visual representations , 1994, CACM.

[31]  Marian Petre,et al.  Why looking isn't always seeing: readership skills and graphical programming , 1995, CACM.

[32]  Lippold Haken,et al.  The Tilia Music Representation: Extensibility, Abstraction, and Notation Contexts for the Lime Music Editor , 1993 .

[33]  Lippold Haken,et al.  A New Algorithm for Horizontal Spacing of Printed Music , 1995, ICMC.

[34]  Philip A. Chou,et al.  Stochastic attribute grammar model of document production and its use in document image decoding , 1995, Electronic Imaging.

[35]  Philip A. Chou,et al.  Document Image Decoding Using Markov Source Models , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Henry S. Baird,et al.  A Critical Survey of Music Image Analysis , 1992 .