3D Multistroke Mapping (3DMM): Transfer of Hand-Drawn Pattern Representation for Skeleton-Based Gesture Recognition

Exergames involve using the fullbody to interact with an immersive world, which raises the challenge of capturing, processing and recognizing the action of the user even for cheap mocap systems such as the Microsoft Kinect. In fact, these recent technological advances have renewed interest in skeleton-based action recognition. Our review of related literature reveals that the issues encountered are not the result of random processes, which could simply be studied by using statistical tools, but are instead due to the fact that the pattern to be recognized, i.e. an action, was produced by a human being. 2D hand-drawn symbols are further examples of patterns resulting from a human motion. Therefore, the main contribution of this paper is to examine the validity of transferring the expertise of hand-drawn symbol representation to better recognize actions based on skeleton data. Principally, we propose a new action representation, namely the 3DMM, as an initial case-study illustrating how such transfer could be conducted. The experimental results, obtained over two benchmarks, confirm the soundness of our approach and encourage more thorough examination of the transfer.