Recognition of Pen-Based Music Notation: The HOMUS Dataset

A profitable way of digitizing a new musical composition is by using a pen-based (online) system, in which the score is created with the sole effort of the composition itself. However, the development of such systems is still largely unexplored. Some studies have been carried out but the use of particular little datasets has led to avoid objective comparisons between different approaches. To solve this situation, this work presents the Handwritten Online Musical Symbols (HOMUS) dataset, which consists of 15200 samples of 32 types of musical symbols from 100 different musicians. Several alternatives of recognition for the two modalities -online, using the strokes drawn by the pen, and offline, using the image generated after drawing the symbol- are also presented. Some experiments are included aimed to draw main conclusions about the recognition of these data. It is expected that this work can establish a binding point in the field of recognition of online handwritten music notation and serve as a baseline for future developments.

[1]  Alex Waibel,et al.  Readings in speech recognition , 1990 .

[2]  S. Eddy.  Hidden Markov models. , 1996, Current opinion in structural biology.

[3]  Daniel Graupe,et al.  Principles of Artificial Neural Networks - 2nd Edition , 2007, Advanced Series in Circuits and Systems.

[4]  Kian Chin Lee,et al.  Handwritten music notation recognition using HMM — a non-gestural approach , 2010, 2010 International Conference on Information Retrieval & Knowledge Management (CAMP).

[5]  Jaime S. Cardoso,et al.  Optical recognition of music symbols , 2010, International Journal on Document Analysis and Recognition (IJDAR).

[6]  Verónica Romero,et al.  Interactive Off-Line Handwritten Text Transcription Using On-Line Handwritten Text as Feedback , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[7]  Timothy C. Bell,et al.  The Challenge of Optical Music Recognition , 2001, Comput. Humanit..

[8]  Norbert Link,et al.  Gesture recognition with inertial sensors and optimized DTW prototypes , 2010, 2010 IEEE International Conference on Systems, Man and Cybernetics.

[9]  Susan E. George,et al.  Online Pen-Based Recognition of Music Notation with Artificial Neural Networks , 2003, Computer Music Journal.

[10]  Marcos Faúndez-Zanuy,et al.  On-line signature recognition based on VQ-DTW , 2007, Pattern Recognit..

[12]  J. Anstice,et al.  The design of a pen-based musical input system , 1996, Proceedings Sixth Australian Conference on Computer-Human Interaction.

[13]  Ichiro Fujinaga,et al.  A Comparative Study of Staff Removal Algorithms , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Frederick Jelinek,et al.  Statistical methods for speech recognition , 1997 .

[15]  Lei Hu,et al.  HMM-Based Recognition of Online Handwritten Mathematical Symbols Using Segmental K-Means Initialization and a Modified Pen-Up/Down Feature , 2011, 2011 International Conference on Document Analysis and Recognition.

[16]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[17]  Herbert Freeman,et al.  On the Encoding of Arbitrary Geometric Configurations , 1961, IRE Trans. Electron. Comput..

[18]  Éric Anquetil,et al.  A generic method to design pen-based systems for structured document composition : Development of a musical score editor , 2005 .

[19]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[20]  Hany Ahmed,et al.  Combining online and offline systems for Arabic handwriting recognition , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[21]  J.A. Anderson,et al.  Neurocomputing: Foundations of Research@@@Neurocomputing 2: Directions for Research , 1992 .

[22]  Carlos Guedes,et al.  Optical music recognition: state-of-the-art and open issues , 2012, International Journal of Multimedia Information Retrieval.

[23]  Laurent Pugin,et al.  Optical Music Recognitoin of Early Typographic Prints using Hidden Markov Models , 2006, ISMIR.

[24]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[25]  Vladimir I. Levenshtein,et al.  Binary codes capable of correcting deletions, insertions, and reversals , 1965 .

[26]  Pavel Slavík,et al.  Music alphabet for low-resolution touch displays , 2009, Advances in Computer Entertainment Technology.

[27]  María José del Jesús,et al.  KEEL: a software tool to assess evolutionary algorithms for data mining problems , 2008, Soft Comput..

[28]  Daniel Graupe,et al.  Principles of Artificial Neural Networks , 2018, Advanced Series in Circuits and Systems.

[29]  Minoru Maruyama,et al.  An online handwritten music symbol recognition system , 2007, International Journal of Document Analysis and Recognition (IJDAR).