Handwritten music notation recognition using HMM — a non-gestural approach

We investigate the recognition of handwritten musical notation using Hidden Markov Models (HMMs). In a non-gestural approach, handwritten musical notation is entered naturally via a pen tablet as we would do using pen and paper. A sequence of observed ink patterns representing musical symbols is captured and used to construct different HMM models. The proposed approach exploits both global and local information derived from ink patterns which we have demonstrated the exploitation of this information via different features employed in different HMMs. The specificity and sensitivity measures of these different classification models are compared using unseen test sets. The experiment shows that using non-gestural method is a very good approach to obtain handwritten music notation input, as it is most natural and does not require user training. It is also shown that HMM is very suitable to be used as the classifier in this domain, showing very high recognition rates. Additionally, the experimental results also concluded that models from HMMs with more hidden states also outperform the HMM with a lesser number of hidden states since a larger model has more capacity. The results also suggest that HMMs offer flexibility in encoding useful knowledge in the models.

[1]  Andrew R. Brown Composing by Pen: Exploring the effectiveness of a system for writing music notation on computers via pen input , 1998 .

[2]  Jianying Hu,et al.  HMM Based On-Line Handwriting Recognition , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Minoru Maruyama,et al.  An online handwritten music symbol recognition system , 2007, International Journal of Document Analysis and Recognition (IJDAR).

[4]  Jean-François Allouis Use of High-Speed Microprocessors for Digital Synthesis , 1978, ICMC.

[5]  G. Lorette,et al.  On-line cursive handwriting analysis and recognition: paradigm and systems , 1994 .

[6]  Ramjee Prasad,et al.  Hidden Markov models applied to on-line handwritten isolated character recognition , 1994, IEEE Trans. Image Process..

[7]  Florence Rossant,et al.  A global method for music symbol recognition in typeset music sheets , 2002, Pattern Recognit. Lett..

[8]  Ivan E. Sutherland,et al.  Sketch pad a man-machine graphical communication system , 1964, DAC.

[9]  Ivan E. Sutherland,et al.  Sketchpad: a man-machine graphical communication system , 1899, AFIPS '63 (Spring).

[10]  Susan E. George Pen-Based Input for On-Line Handwritten Notation , 2004 .

[11]  Kin Hong Wong,et al.  Script recognition using hidden Markov models , 1986, ICASSP '86. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[12]  Jay J. Lee,et al.  Data-Driven Design of HMM Topology for Online Handwriting Recognition , 2001, Int. J. Pattern Recognit. Artif. Intell..

[13]  Jin Xiangyu,et al.  An online composite graphics recognition approach based on matching of spatial relation graphs , 2004 .

[14]  Wenyin Liu,et al.  An online composite graphics recognition approach based on matching of spatial relation graphs , 2004, Document Analysis and Recognition.

[15]  A. Brakensiek,et al.  Writer adaptation for on-line handwriting recognition , 2001 .

[16]  Andrew S. Forsberg,et al.  The music notepad , 1998, UIST '98.

[17]  JinHyung Kim,et al.  Data-driven Design of HMM Topology for On-line Handwriting Recognition , 2000 .

[18]  Anil K. Jain,et al.  Writer Adaptation for Online Handwriting Recognition , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Éric Anquetil,et al.  A generic method to design pen-based systems for structured document composition : Development of a musical score editor , 2005 .

[20]  J. Anstice,et al.  The design of a pen-based musical input system , 1996, Proceedings Sixth Australian Conference on Computer-Human Interaction.

[21]  Susan E. George,et al.  Online Pen-Based Recognition of Music Notation with Artificial Neural Networks , 2003, Computer Music Journal.

[22]  Dorothea Blostein,et al.  A graph grammar programming style for recognition of music notation , 1993, Machine Vision and Applications.

[23]  Dit-Yan Yeung,et al.  Mathematical expression recognition: a survey , 2000, International Journal on Document Analysis and Recognition.

[24]  Minoru Maruyama,et al.  A fast HMM algorithm based on stroke lengths for on-line recognition of handwritten music scores , 2004, Ninth International Workshop on Frontiers in Handwriting Recognition.

[25]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .