Identity verification using speech and face information

This article first provides an overview of important concept s in the field of information fusion, followed by a review of milestones in audio-visual person identification and verifi cation. Several recent adaptive and non-adaptive techniques for reaching the verification decision (i.e., to accept or re ject the claimant), based on speech and face information, are then evaluated in clean and noisy audio conditions on a common database; it is shown that in clean conditions most of the non-adaptive approaches provide similar performance and in noisy conditions most exhibit a severe deterioration i n performance; it is also shown that current adaptive approac hes are either inadequate or utilize restrictive assumptio ns. A new category of classifiers is then introduced, where the de cision boundary is fixed but constructed to take into account how the distributions of opinions are likely to change due to noisy conditions; compared to a previously proposed adaptive approach, the proposed classifiers do not make a direct assum ption about the type of noise that causes the mismatch between training and testing conditions.