We present a video-based system for tracking 2D face motion during speech. The system tracks dot markers on the speaker's face from video sequences. A digital video camera is used to film the speaker during speech production experiments. The acquired digital video is converted into an image sequence, which is then provided as input to the tracking algorithm. The algorithm locates all dots in every frame of the sequence, and outputs the x and y coordinates of the dots in the images over time. Continuity constraints on the face motion are imposed when searching for the dots. The algorithm was tested on video sequences for both spontaneous and non-spontaneous (i.e., reading) speech conditions, and no mistracking occurred for the sequences used
[1]
E. Vatikiotis-Bateson,et al.
Kinematics-Based Synthesis of Realistic Talking Faces
,
1998,
AVSP.
[2]
Eric Vatikiotis-Bateson,et al.
Target practice on talking faces
,
2004,
INTERSPEECH.
[3]
Hani Yehia,et al.
Quantitative association of vocal-tract and facial behavior
,
1998,
Speech Commun..
[4]
K. G. Munhall,et al.
Spatial frequency requirements for audiovisual speech perception
,
2004,
Perception & psychophysics.
[5]
Takaaki Kuratate,et al.
Video-based face motion measurement
,
2002,
J. Phonetics.
[6]
Hani Yehia,et al.
Speaking mode variability in multimodal speech production
,
2002,
IEEE Trans. Neural Networks.