Robotic Chinese Calligraphy with Human Preference

Robotic Chinese calligraphy is an attempt to lead robots to learn mankind's culture and knowledge. The current research on robotic calligraphy ignores the usage of human preferences. This has restricted robots to produce writing results reflecting personalized styles. This paper proposes a robotic learning approach that introduces a inverse reinforcement learning algorithm with human preferences into a robotic writing system. Through selections of human users, the robot system learns to write Chinese character strokes according to the user's aesthetic preference. Thus, the paper first uses a generative network adopting from the Generative Adversarial Nets to produce a basic writing ability of Chinese strokes for a robot system. Then, the writing results of the robot are captured by the robot's visual device and then presented to the human users as images. Then, the human users give their preferences as the feedbacks of the images, the approach uses the marked images to train a reward predictive mechanism. In the end, the reward predictive mechanism aids the inverse reinforcement learning algorithm to enable the robot to automatically improve its writing ability of Chinese character strokes. Experimental results show that the proposed framework can successfully allow the robot to write Chinese characters strokes in accordance with the human user's preference. In addition, the robot demonstrates a fast learning speed with a small number of human selections. This gives a very promising solution to the robot's learning of more complex movements.