Online trajectory recovery from offline handwritten Japanese kanji characters of multiple strokes

In general, it is straightforward to render an offline handwriting image from an online handwriting pattern. However, it is challenging to reconstruct an online handwriting pattern given an offline handwriting image, especially for multiple-stroke character as Japanese kanji. The multiple-stroke character requires not only point coordinates but also stroke orders whose difficulty is exponential growth by the number of strokes. Besides, several crossed and touch points might increase the difficulty of the recovered task. We propose a deep neural network-based method to solve the recovered task using a large online handwriting database. Our proposed model has two main components: Convolutional Neural Network-based encoder and Long Short-Term Memory Network-based decoder with an attention layer. The encoder focuses on feature extraction while the decoder refers to the extracted features and generates the time-sequences of coordinates. We also demonstrate the effect of the attention layer to guide the decoder during the reconstruction. We evaluate the performance of the proposed method by both visual verification and handwritten character recognition. Although the visual verification reveals some problems, the recognition experiments demonstrate the effect of trajectory recovery in improving the accuracy of offline handwritten character recognition when online recognition for the recovered trajectories are combined.