Motion analysis: Action detection, recognition and evaluation based on motion capture data

Abstract A novel framework, for real-time action detection, recognition and evaluation of motion capture data, is presented in this paper. Pose and kinematics information is used for data description. Automatic and dynamic weighting, altering joint data significance based on action involvement, and Kinetic energy-based descriptor sampling are employed for efficient action segmentation and labelling. The automatically segmented and recognized action instances are subsequently fed to the framework action evaluation component, which compares them with the corresponding reference ones, estimating their similarity. Exploiting fuzzy logic, the framework subsequently gives semantic feedback with instructions on performing the actions more accurately. Experimental results on MSR-Action3D and MSRC12 benchmarking datasets and a new, publicly available one, provide evidence that the proposed framework compares favourably to state-of-the-art methods by 0.5–6% in all three datasets, showing that the proposed method can be effectively used for unsupervised gesture/action training.