Recognition of Online Handwritten Math Symbols Using Deep Neural Networks

This paper presents deep learning to recognize online handwritten mathematical symbols. Recently various deep learning architectures such as Convolution neural networks (CNNs), Deep neural networks (DNNs), Recurrent neural networks (RNNs) and Long short-term memory (LSTM) RNNs have been applied to fields such as computer vision, speech recognition and natural language processing where they have shown superior performance to state-of-the-art methods on various tasks. In this paper, max-out-based CNNs and Bidirectional LSTM (BLSTM) networks are applied to image patterns created from online patterns and to the original online patterns, respectively and then combined. They are compared with traditional recognition methods which are MRFs and MQDFs by recognition experiments on the CROHME database along with analysis and explanation. key words: CNN, BLSTM, gradient features, dropout, maxout

[1]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[2]  Joan-Andreu Sánchez,et al.  Classification of On-Line Mathematical Symbols with Hybrid Features and Recurrent Neural Networks , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[3]  Masaki Nakagawa,et al.  Online Handwritten Chinese/Japanese Character Recognition , 2012 .

[4]  Masaki Nakagawa,et al.  Effects of Line Densities on Nonlinear Normalization for Online Handwritten Japanese Character Recognition , 2011, 2011 International Conference on Document Analysis and Recognition.

[5]  Alex Graves,et al.  Supervised Sequence Labelling , 2012 .

[6]  Joan-Andreu Sánchez,et al.  Offline Features for Classifying Handwritten Math Symbols with Recurrent Neural Networks , 2014, 2014 22nd International Conference on Pattern Recognition.

[7]  Yoshua Bengio,et al.  Maxout Networks , 2013, ICML.

[8]  Fumitaka Kimura,et al.  Modified Quadratic Discriminant Functions and the Application to Chinese Character Recognition , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[10]  Ching Y. Suen,et al.  A Method of Combining Multiple Experts for the Recognition of Unconstrained Handwritten Numerals , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Jinyu Li,et al.  Investigation of maxout networks for speech recognition , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[14]  K. Ishigaki,et al.  Hybrid pen-input character recognition system based on integration of online-offline recognition , 1999, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99 (Cat. No.PR00318).

[15]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[16]  A. Tanaka,et al.  Online recognition of freely handwritten Japanese characters using directional feature densities , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[17]  Harold Mouchère,et al.  ICFHR 2014 Competition on Recognition of On-Line Handwritten Mathematical Expressions (CROHME 2014) , 2014, 2014 14th International Conference on Frontiers in Handwriting Recognition.

[18]  Masaki Nakagawa,et al.  A MRF model with parameter optimization by CRF for on-line recognition of handwritten Japanese characters , 2011, Electronic Imaging.

[19]  Urs Ramer,et al.  An iterative procedure for the polygonal approximation of plane curves , 1972, Comput. Graph. Image Process..