Generalized photometric stereo and its application to face recognition

Most photometric stereo algorithms employ a Lambertian reflectance mode l with a varying albedo field and involve the appearances of only one object. Th e recovered albedos and surface normals are object-specific and appearances not belonging to the object cannot be easily handled. We generalize photometric stereo algorithms to handle all appearances of all objects in a class, in particular the human face class, by assuming tha t albedos and surface normals of all objects in the class be rank-constrained, i.e. lie in a subspace. Rank constraints lead us to a factorization of an observation matrix that consists o f exemplar images of different objects under different illuminations. To fully recover the s ubspace bases or class- specific albedos and surface normals, we employ integrability and face symmetry constraints and propose a linearized algorithm. This algorithm takes into account the effects of the varying albedo field by approximating the integrability terms using only the surface no rmals. We then apply our generalized photometric stereo algorithm for recognizing faces under illumination variations. As far as recognition is concerned, we can utilize a bootstrap s et which is just a collection of 2D image observations to avoid an explicit requirement that 3D information be available. We obtain good recognition results using the PIE database.