High-fidelity facial performance capture with non-sequential temporal alignment

The 3D capture of facial performance has become a necessary tool for delivering visual effects related to an actor's face or for creating new realistic characters. We present a novel system using a practical acquisition setup without active illumination or high-speed recording. Non-sequential surface tracking based on the minimum spanning tree is introduced for the facial performance application. This optimal traversal through the sequence is combined with a frame-to-frame alignment method which is robust against weak texture and fast non-rigid motions. Photometric stereo based on colour illumination reconstructs metrically correct normals of fine skin geometry. The resulting 3D model consists of the temporally consistent mesh sequence combined with high-detail UV normal maps.