Face Verification Using Synthesized Non-Frontal Models

{NOTE}: {THIS} {REPORT} {HAS} {BEEN} {SUPERSEDED} {BY} {IDIAP-RR} 04-04. {I}n this report we address the problem of non-frontal face verification when only a frontal training image is available (e.g. a passport photograph) by augmenting a client's frontal face model with artificially synthesized models for non-frontal views. In the framework of a {G}aussian {M}ixture {M}odel ({GMM}) based classifier, two techniques are proposed for the synthesis: {UBM}diff and {L}in{R}eg. {B}oth techniques rely on prior information and learn how face models for the frontal view are related to face models at a non-frontal view. {T}he synthesis and augmentation approach is evaluated by applying it to two face verification systems: {P}rincipal {C}omponent Analysis ({PCA}) based and {DCT}mod2 based; the two systems are a representation of holistic and non-holistic approaches, respectively. {R}esults from experiments on the {FERET} database suggest that in almost all cases, frontal model augmentation has beneficial effects for both systems; they also suggest that the {L}in{R}eg technique (which is based on multivariate regression of classifier parameters) is more suited to the {PCA} based system and that the {UBM}diff technique (which is based on differences between two general face models) is more suited to the {DCT}mod2 based system. The results also support the view that the standard {DCT}mod2/{GMM} system (trained on frontal faces) is less affected by out-of-plane rotations than the corresponding {PCA}/{GMM} system; moreover, the {DCT}mod2/{GMM} system using augmented models is, in almost all cases, more robust than the corresponding {PCA}/{GMM} system.