Incorporating Illumination Constraints in Deformable Models for Shape from Shading and Light Direction Estimation

We present a method for the integration of nonlinear holonomic constraints in deformable models and its application to the problems of shape and illuminant direction estimation from shading. Experimental results demonstrate that our method performs better than previous Shape from Shading algorithms applied to images of Lambertian objects under known illumination. It is also more general as it can be applied to non-Lambertian surfaces and it does not require knowledge of the illuminant direction. In this paper, (1) we first develop a theory for the numerically robust integration of nonlinear holonomic constraints within a deformable model framework. In this formulation, we use Lagrange multipliers and a Baumgarte stabilization approach (1972). (2) We also describe a fast new method for the computation of constraint based forces, in the case of high numbers of local parameters. (3) We demonstrate how any type of illumination constraint, from the simple Lambertian model to more complex highly nonlinear models can be incorporated in a deformable model framework. (4) We extend our method to work when the direction of the light source is not known. We couple our shape estimation method with a method for light estimation, in an iterative process, where improved shape estimation results in improved light estimation and vice versa. (5) We perform a series of experiments.

[1]  Elli Angelopoulou,et al.  Photometric surface analysis in a tri-luminal environment , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[2]  Dimitris N. Metaxas,et al.  Coupled lighting direction and shape estimation from single images , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[3]  Shree K. Nayar,et al.  Reflectance and texture of real-world surfaces , 1999, TOGS.

[4]  Alex Pentland.  Linear shape from shading , 2004, International Journal of Computer Vision.

[5]  E. Rouy,et al.  A viscosity solutions approach to shape-from-shading , 1992 .

[6]  Alex Pentland,et al.  Shape Information From Shading: A Theory About Human Perception , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[7]  Rama Chellappa,et al.  A Method for Enforcing Integrability in Shape from Shading Algorithms , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Robert J. Woodham,et al.  Analysing Images of Curved Surfaces , 1981, Artif. Intell..

[9]  Rama Chellappa,et al.  Estimation of illuminant direction, albedo, and shape from shading , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  David J. Kriegman,et al.  Illumination-based image synthesis: creating novel images of human faces under differing pose and lighting , 1999, Proceedings IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes (MVIEW'99).

[11]  J. Wittenburg,et al.  Dynamics of systems of rigid bodies , 1977 .

[12]  Dimitris N. Metaxas.  Physics-Based Deformable Models: Applications to Computer Vision, Graphics, and Medical Imaging , 1996 .

[13]  Aaron F. Bobick,et al.  The direct computation of height from shading , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Berthold K. P. Horn.  Height and gradient from shading , 1989, International Journal of Computer Vision.

[15]  A. Pentland.  Shape information from shading: A theory about human perception , 1989 .

[16]  Shree K. Nayar,et al.  Generalization of the Lambertian model and implications for machine vision , 1995, International Journal of Computer Vision.

[17]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[19]  Pascal Fua,et al.  Variable albedo surface reconstruction from stereo and shape from shading , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[20]  Edwin R. Hancock,et al.  New Constraints on Data-Closeness and Needle Map Consistency for Shape-from-Shading , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Alfred M. Bruckstein,et al.  Global Shape from Shading , 1996, Comput. Vis. Image Underst..

[22]  David J. Kriegman,et al.  What is the set of images of an object under all possible lighting conditions? , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Dimitris N. Metaxas,et al.  Shape and Nonrigid Motion Estimation Through Physics-Based Synthesis , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Mubarak Shah,et al.  A fast linear shape from shading , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  Richard Szeliski,et al.  Fast shape from shading , 1990, CVGIP Image Underst..

[26]  John Oliensis,et al.  A global algorithm for shape from shading , 1993, 1993 (4th) International Conference on Computer Vision.

[27]  Dimitris N. Metaxas,et al.  The integration of optical flow and deformable models with applications to human face shape and motion estimation , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Takeo Kanade,et al.  Surface Reflection: Physical and Geometrical Perspectives , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Gilbert Strang,et al.  Introduction to applied mathematics , 1988 .

[30]  Azriel Rosenfeld,et al.  Improved Methods of Estimating Shape from Shading Using the Light Source Coordinate System , 1985, Artif. Intell..

[31]  Dimitris N. Metaxas,et al.  Incorporating illumination constraints in deformable models , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[32]  Michael J. Brooks,et al.  Shape and Source from Shading , 1985, IJCAI.

[33]  Alex Pentland,et al.  Local Shading Analysis , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Johan Montagnat,et al.  New Algorithms for Controlling Active Contours Shape and Topology , 2000, ECCV.

[35]  Berthold K. P. Horn.  SHAPE FROM SHADING: A METHOD FOR OBTAINING THE SHAPE OF A SMOOTH OPAQUE OBJECT FROM ONE VIEW , 1970 .

[36]  Pascal Fua,et al.  Imposing Hard Constraints on Deformable Models through Optimization in Orthogonal Subspaces , 1997, Comput. Vis. Image Underst..

[37]  Shree K. Nayar,et al.  Reflectance and Texture of Real-World Surfaces Authors , 1997, CVPR 1997.

[38]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[39]  C.-C. Jay Kuo,et al.  Shape from Shading with a Linear Triangular Element Surface Model , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Pascal Fua,et al.  From Regular Images to Animated Heads: A Least Squares Approach , 1998, ECCV.

[41]  Katsushi Ikeuchi,et al.  Numerical Shape from Shading and Occluding Boundaries , 1981, Artif. Intell..

[42]  J. Baumgarte.  Stabilization of constraints and integrals of motion in dynamical systems , 1972 .

[43]  Pascal Fua,et al.  Object-centered surface reconstruction: Combining multi-image stereo and shading , 1995, International Journal of Computer Vision.

[44]  Alex Pentland,et al.  A simple algorithm for shape from shading , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[45]  William H. Press,et al.  Numerical recipes in C , 2002 .

[46]  Ron Kimmel,et al.  An optimal time algorithm for shape from shading , 1998 .