Statistics-based face rendering and its application to face recognition

Realistic rendering of human faces and robust face recognition are two topics that have wide applicability, ranging from video conferencing to entertainment to security systems. Consequently they have received much research attention in recent years. Many proposed techniques have advanced the state of the art: in terms of rendering, the synthesized face images are of such good quality as to be virtually indistinguishable from an actual photograph; while in terms of face recognition, current systems work well enough to be deployed at airports or city centers. Yet there are still a number of deficiencies. On the one hand, many rendering techniques are ad hoc, lacking in theoretical justification. For instance, there is no quantifiable way to know if the rendered image is optimal. On the other hand, face recognition systems are not robust enough to deal with simultaneous changes in illumination, head pose, etc. Finding a unifying approach to deal with this problem appears to be elusive. In this dissertation, we introduce an approach for rendering faces that is principled, statistically optimal, and that possesses several good theoretical properties. A unique feature of our approach is that we regard the shape of the face as an intermediate variable only, and do not explicitly recover it in the rendering process. We also present a new approach to face recognition that takes into account the (limited) training data, that is robust against illumination variation, and that has the potential for robustness against other kinds of variation.