Combining marker-based mocap and RGB-D camera for acquiring high-fidelity hand motion data

Motion capture data has been pivotal to the success of creating realistic animation for human characters. There are a number of public full-body motion databases available, but large and heterogeneous databases for hand articulations are not available. In this paper, we introduce a novel acquisition framework for acquiring a wide range of high-fidelity hand motion data. Our key idea is to leverage marker position data recorded by a twelve-camera optical motion capture system and RGB/Depth data obtained from a single Microsoft Kinect camera. We formulate the hand motion capture problem in a nonlinear optimization framework by maximizing consistency between the reconstructed motion and observed measurement. We introduce an efficient optimization technique to estimate the optimal hand pose that best matches observed data. We have demonstrated the power and effectiveness of our system by capturing a wide variety of delicate hand articulations, even in case of significant self-occlusion.