Fusion Networks for Air-Writing Recognition

This paper presents a fusion framework for air-writing recognition. By modeling a hand trajectory using both spatial and temporal features, the proposed network can learn more information than the state-of-the-art techniques. The proposed network combines elements of CNN and BLSTM networks to learn the isolated air-writing characters. The performance of proposed network was evaluated by the alphabet and numeric databases in the public dataset namely 6DMG. We first evaluate the accuracy of fusion network using CNN, BLSTM, and another fusion network as the references. The results confirmed that the average accuracy of fusion network outperforms all of the references. When the BLSTM unit was set at 40, the best accuracy of proposed network is 99.27% and 99.33% in the alphabet and numeric gesture, respectively. When compared this result with another work, the accuracy of proposed network improves 0.70% and 0.34% in the alphabet and numeric gesture, respectively. We also examine the performance of the proposed network by varying the number of BLSTM units. The experiments demonstrate that while increasing the number of BLSTM units, the accuracy also improves. When the BLSTM unit is greater than 20, the accuracy maintains even though the BLSTM unit increases. Despite adding more learning features, the accuracy of proposed network insignificantly improves.

[1]  Debi Prosad Dogra,et al.  Segmentation and recognition of text written in 3D using Leap motion interface , 2015, 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR).

[2]  Volkmar Frinken,et al.  Deep BLSTM neural networks for unconstrained continuous handwritten text recognition , 2015, 2015 13th International Conference on Document Analysis and Recognition (ICDAR).

[3]  Yang Xue,et al.  Air-writing characters modelling and recognition on modified CHMM , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[4]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[5]  Alexander H. Waibel,et al.  Online handwriting recognition: the NPen++ recognizer , 2001, International Journal on Document Analysis and Recognition.

[6]  Bonhwa Ku,et al.  Alpha-numeric hand gesture recognition based on fusion of spatial feature modelling and temporal feature modelling , 2016 .

[7]  Y. LeCun.  Neural Networks and Gradient-Based Learning in OCR , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[8]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[9]  Yoshua Bengio,et al.  Drawing and Recognizing Chinese Characters with Recurrent Neural Network , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Yue Ming,et al.  Trajectory image based dynamic gesture recognition with convolutional neural networks , 2015, 2015 15th International Conference on Control, Automation and Systems (ICCAS).

[11]  Andreas Holzinger,et al.  On Using Entropy for Enhancing Handwriting Preprocessing , 2012, Entropy.

[12]  Guillermo Garcia-Hernando,et al.  Novel spatio-temporal features for fingertip writing recognition in egocentric viewpoint , 2015, 2015 14th IAPR International Conference on Machine Vision Applications (MVA).

[13]  Biing-Hwang Juang,et al.  6DMG: a new 6D motion gesture database , 2012, MMSys '12.

[14]  Jürgen Schmidhuber,et al.  Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks , 2006, ICML.

[15]  Yu-Liang Hsu,et al.  An Inertial Pen With Dynamic Time Warping Recognizer for Handwriting and Gesture Recognition , 2015, IEEE Sensors Journal.

[16]  J. Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM networks , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[17]  Jing Zhang,et al.  Modified CRF algorithm for dynamic hand gesture recognition , 2014, Proceedings of the 33rd Chinese Control Conference.