Lip localization and performance evaluation

Although impressive achievement has been made in the field of lip localization, there is no clear picture showing that which algorithms or methods have technical advantages over others. It is partly due to the fact that the commonly agreed evaluation methodology has not been established yet in the field. The common practice is using manual lip labelling as the ground truth for evaluation marked by human labellers. Our empirical results using the Expectation-Maximization procedure show that human labeller introduce subjective noises while labelling the lip-localization and often the subjective noises added can be too significant to be directly used for evaluation as ground truth. We argue in the paper that to train and evaluate a lip analysis system one has to measure the quality of human operators and infer the "ground truth" from the manual labelling, simultaneously. We demonstrate using BioID database how the Expectation- Maximization technology can be used for this purpose.