Variable albedo surface reconstruction from stereo and shape from shading

We present a multiview method for the computation of object shape and reflectance characteristics based on the integration of shape from shading (SFS) and stereo, for nonconstant albedo and non-uniformly Lambertian surfaces. First we perform stereo fitting on the input stereo pairs or image sequences. When the images are uncalibrated, we recover the camera parameters using bundle adjustment. Eased on the stereo result, we can automatically segment the albedo map (which is taken to be piece-wise constant) using a minimum description length (MDL) based metric, to identify areas suitable for SFS (typically smooth textureless areas) and to derive illumination information. The shape and the illumination parameter estimates are refined using a deformable model SFS algorithm, which iterates between computing shape and illumination parameters. Our method takes into account the viewing angle dependent for shortening and specularity effects, and compensates as much as possible by utilizing information from more than one images. We demonstrate that we can extend the applicability of SFS algorithms to real world situations when some of its traditional assumptions are violated. We demonstrate our method by applying it to face shape reconstruction. Experimental results indicate a significant improvement over SFS-only or stereo-only based reconstruction. Model accuracy and detail are improved, especially in areas of low texture detail. Albedo information is retrieved and can be used to accurately re-render the model under different illumination conditions.

[1]  Dimitris N. Metaxas,et al.  Coupled lighting direction and shape estimation from single images , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[2]  Dimitris N. Metaxas.  Physics-Based Deformable Models: Applications to Computer Vision, Graphics, and Medical Imaging , 1996 .

[3]  Dimitris N. Metaxas,et al.  Incorporating illumination constraints in deformable models , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[4]  Pascal Fua,et al.  From Regular Images to Animated Heads: A Least Squares Approach , 1998, ECCV.

[5]  William H. Press,et al.  Numerical recipes in C , 2002 .

[6]  Shree K. Nayar,et al.  Diffuse reflectance from rough surfaces , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Pascal Fua,et al.  Using model-driven bundle-adjustment to model heads from raw video sequences , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[8]  David Salesin,et al.  Synthesizing realistic facial expressions from photographs , 1998, SIGGRAPH.

[9]  Daniel Thalmann,et al.  Simulation of Facial Muscle Actions Based on Rational Free Form Deformations , 1992, Comput. Graph. Forum.

[10]  Aaron F. Bobick,et al.  The direct computation of height from shading , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Mubarak Shah,et al.  Analysis of shape from shading techniques , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[13]  J. Baumgarte.  Stabilization of constraints and integrals of motion in dynamical systems , 1972 .

[14]  Luc Van Gool,et al.  Active acquisition of 3D shape for moving objects , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[15]  Pascal Fua,et al.  Combining Stereo and Monocular Information to Compute Dense Depth Maps that Preserve Depth Discontinuities , 1991, IJCAI.

[16]  Pascal Fua,et al.  Characterizing the Performance of Multiple-Image Point-Correspondence Algorithms Using Self-Consistency , 1999, Workshop on Vision Algorithms.

[17]  Mubarak Shah,et al.  Integration of shape from shading and stereo , 1995, Pattern Recognit..

[18]  Samuel Kotz,et al.  Encyclopedia of Statistical Sciences (supp.). , 1990 .

[19]  Stephen H. Westin,et al.  Reflectance Measurements of Human Skin , 1999 .