Online recognition of handwritten music symbols

In this paper, we propose an effective online method to recognize handwritten music symbols. Based on the fact that most music symbols can be regarded as combinations of several basic strokes, the proposed method first classifies all the strokes comprising an input symbol and then recognizes the symbol based on the results of stroke classification. For stroke classification, we propose to use three types of features, which are the size information, the histogram of directional movement angles, and the histogram of undirected movement angles. When combining classified strokes into a music symbol, we utilize their sizes and spatial relation together with their combination. The proposed method is evaluated using two datasets including HOMUS, one of the largest music symbol datasets. As a result, it achieves a significant improvements of about 10% in recognition rates compared to the state-of-the-art method for the datasets. This shows the superiority of the proposed method in online handwritten music symbol recognition.

[1]  Minoru Maruyama,et al.  An online handwritten music symbol recognition system , 2007, International Journal of Document Analysis and Recognition (IJDAR).

[2]  Herbert Freeman,et al.  On the Encoding of Arbitrary Geometric Configurations , 1961, IRE Trans. Electron. Comput..

[3]  Sargur N. Srihari,et al.  On-Line and Off-Line Handwriting Recognition: A Comprehensive Survey , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Carlos Guedes,et al.  Optical music recognition: state-of-the-art and open issues , 2012, International Journal of Multimedia Information Retrieval.

[5]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[6]  José Oncina,et al.  Clustering of Strokes from Pen-Based Music Notation: An Experimental Study , 2015, IbPRIA.

[7]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[8]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[9]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[10]  Kian Chin Lee,et al.  Handwritten music notation recognition using HMM — a non-gestural approach , 2010, 2010 International Conference on Information Retrieval & Knowledge Management (CAMP).

[11]  Lei Hu,et al.  HMM-Based Recognition of Online Handwritten Mathematical Symbols Using Segmental K-Means Initialization and a Modified Pen-Up/Down Feature , 2011, 2011 International Conference on Document Analysis and Recognition.

[12]  Horst Bunke,et al.  Applications of approximate string matching to 2D shape recognition , 1993, Pattern Recognit..

[13]  Jorge Calvo-Zaragoza,et al.  Recognition of online handwritten music symbols , 2013 .

[14]  José Oncina,et al.  Recognition of Pen-Based Music Notation: The HOMUS Dataset , 2014, 2014 22nd International Conference on Pattern Recognition.

[15]  Claus Bahlmann,et al.  Directional features in online handwriting recognition , 2006, Pattern Recognit..

[16]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[17]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[18]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[19]  Zhen-Long Bai,et al.  A study on the use of 8-directional features for online handwritten Chinese character recognition , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[20]  Kazuhiko Yamamoto,et al.  On-line handwriting character recognition using direction-change features that consider imaginary strokes , 1999, Pattern Recognit..

[21]  Jaime S. Cardoso,et al.  Optical recognition of music symbols , 2010, International Journal on Document Analysis and Recognition (IJDAR).

[22]  Susan E. George,et al.  Online Pen-Based Recognition of Music Notation with Artificial Neural Networks , 2003, Computer Music Journal.