Variable-Source Shading Analysis

The shading on curved surfaces is a cue to shape. Current computer vision methods for analyzing shading use physically unrealistic models, have serious mathematical problems, cannot exploit geometric information if it is available, and are not reliable in practice. We introduce a novel method of accounting for variations in irradiance resulting from interreflections, complex sources and the like. Our approach uses a spatially varying source model with a local shading model. Fast spatial variation in the source is penalised, consistent with the rendering community’s insight that interreflections are spatially slow. This yields a physically plausible shading model. Because modern cameras can make accurate reports of observed radiance, our method compels the reconstructed surface to have shading exactly consistent with that of the image. For inference, we use a variational formulation, with a selection of regularization terms which guarantee that a solution exists. Our method is evaluated on physically accurate renderings of virtual objects, and on images of real scenes, for a variety of different kinds of boundary condition. Reconstructions for single sources compare well with photometric stereo reconstructions and with ground truth.

[1]  HighWire Press Philosophical Transactions of the Royal Society of London , 1781, The London Medical Journal.

[2]  Berthold K. P. Horn SHAPE FROM SHADING: A METHOD FOR OBTAINING THE SHAPE OF A SMOOTH OPAQUE OBJECT FROM ONE VIEW , 1970 .

[3]  Berthold K. P. Horn,et al.  Determining lightness from an image , 1974, Comput. Graph. Image Process..

[4]  Berthold K. P. Horn Understanding Image Intensities , 1977, Artif. Intell..

[5]  E. Kreyszig Introductory Functional Analysis With Applications , 1978 .

[6]  Katsushi Ikeuchi,et al.  Numerical Shape from Shading and Occluding Boundaries , 1981, Artif. Intell..

[7]  Dana H. Ballard,et al.  Computer Vision , 1982 .

[8]  Anna R. Bruss The Eikonal equation: some results applicable to computer vision , 1982 .

[9]  A. Pentland Finding the illuminant direction , 1982 .

[10]  Donald P. Greenberg,et al.  Modeling the interaction of light between diffuse surfaces , 1984, SIGGRAPH.

[11]  Andrew Blake,et al.  Surface descriptions from stereo and shading , 1986, Image Vis. Comput..

[12]  Andrew Blake,et al.  Boundary conditions for lightness computation in Mondrian World , 1985, Comput. Vis. Graph. Image Process..

[13]  Michael J. Brooks,et al.  The variational approach to shape from shading , 1986, Comput. Vis. Graph. Image Process..

[14]  Andrew Blake,et al.  Detecting Specular Reflections Using Lambertian Constraints , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[15]  David A. Forsyth,et al.  Mutual illumination , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  David A. Forsyth,et al.  Shape from shading in the light of mutual illumination , 1990, Image and Vision Computing.

[17]  David A. Forsyth,et al.  Reflections on Shading , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  E. Rouy,et al.  A viscosity solutions approach to shape-from-shading , 1992 .

[19]  Richard Szeliski,et al.  Impossible Shaded Images , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Michael F. Cohen,et al.  Radiosity and realistic image synthesis , 1993 .

[21]  Mubarak Shah,et al.  Shape from shading using linear approximation , 1994, Image Vis. Comput..

[22]  Claude Puech,et al.  Radiosity and global illumination , 1994 .

[23]  Jean Ponce,et al.  Recovering the shape of polyhedra using line-drawing analysis and complex reflectance models , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Maurizio Falcone,et al.  An Algorithm for the Global Solution of the Shape-from-Shading Model , 1997, ICIAP.

[25]  G. W. Larson,et al.  Rendering with radiance - the art and science of lighting visualization , 2004, Morgan Kaufmann series in computer graphics and geometric modeling.

[26]  A. Fitzgibbon,et al.  Pictorial relief - Discussion , 1998 .

[27]  Jan J. Koenderink,et al.  Pictorial relief , 2019, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[28]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[29]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Jean-Denis Durou,et al.  Creation of real images which are valid for the assumptions made in shape from shading , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[31]  Edwin R. Hancock,et al.  New Constraints on Data-Closeness and Needle Map Consistency for Shape-from-Shading , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Shree K. Nayar,et al.  Radiometric self calibration , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[33]  David J. Kriegman,et al.  From few to many: generative models for recognition under variable pose and illumination , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[34]  P. Daniel,et al.  From Deterministic to Stochastic Methods for Shape From Shading , 2000 .

[35]  Taku Komura,et al.  Topology matching for fully automatic similarity estimation of 3D shapes , 2001, SIGGRAPH.

[36]  Pat Hanrahan,et al.  A signal-processing framework for inverse rendering , 2001, SIGGRAPH.

[37]  J T Todd,et al.  Ambiguity and the ‘Mental Eye’ in Pictorial Relief , 2001, Perception.

[38]  Ramesh Raskar,et al.  Projector-based three dimensional graphics , 2002 .

[39]  Olivier D. Faugeras,et al.  Shape from Shading and Viscosity Solutions , 2002, ECCV.

[40]  Philippe Bekaert,et al.  Advanced global illumination , 2006 .

[41]  M. Bar A Cortical Mechanism for Triggering Top-Down Facilitation in Visual Object Recognition , 2003, Journal of Cognitive Neuroscience.

[42]  John Oliensis,et al.  Uniqueness in shape from shading , 1991, International Journal of Computer Vision.

[43]  Wojciech Chojnacki,et al.  Impossible and ambiguous shading patterns , 2004, International Journal of Computer Vision.

[44]  Daniel Cremers,et al.  Shedding light on stereoscopic segmentation , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[45]  Greg Humphreys,et al.  Physically Based Rendering: From Theory to Implementation , 2004 .

[46]  Takashi Matsuyama,et al.  Shape from Shading with Interreflections Under a Proximal Light Source: Distortion-Free Copying of an Unfolded Book , 1997, International Journal of Computer Vision.

[47]  Takeo Kanade,et al.  Shape from interreflections , 2004, International Journal of Computer Vision.

[48]  M. Bar Visual objects in context , 2004, Nature Reviews Neuroscience.

[49]  Yehezkel Yeshurun,et al.  Shape-from-Shading Under Perspective Projection , 2005, International Journal of Computer Vision.

[50]  M. Bar,et al.  Can a gun prime a hairbrush? The “initial guesses” that drive top-down contextual facilitation of object recognition , 2005 .

[51]  Olivier D. Faugeras,et al.  A Generic and Provably Convergent Shape-from-Shading Method for Orthographic and Pinhole Cameras , 2005, International Journal of Computer Vision.

[52]  Hany Farid,et al.  Exposing digital forgeries by detecting inconsistencies in lighting , 2005, MM&Sec '05.

[53]  David A. Forsyth,et al.  Fast and detailed approximate global illumination by irradiance decomposition , 2005, ACM Trans. Graph..

[54]  Olivier D. Faugeras,et al.  Shape from shading: a well-posed problem? , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[55]  Kadi Bouatouch,et al.  Radiance caching for efficient global illumination computation , 2005 .

[56]  Edwin R. Hancock,et al.  Surface radiance correction for shape from shading , 2005, Pattern Recognit..

[57]  David J. Kriegman,et al.  Integrating Surface Normal Vectors Using Fast Marching Method , 2006, ECCV.

[58]  A ray tracing solution for diffuse interreflection , 1988, SIGGRAPH '07.

[59]  Brian Potetz,et al.  Efficient Belief Propagation for Vision Using Linear Constraint Nodes , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[60]  J. Koenderink,et al.  Ambiguity in Pictorial Depth , 2007, Perception.

[61]  Antonio Robles-Kelly,et al.  Shape-From-Shading Using the Heat Equation , 2007, IEEE Transactions on Image Processing.

[62]  David J. Kriegman,et al.  ShadowCuts: Photometric Stereo with Shadows , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[63]  Roberto Cipolla,et al.  Shadows in Three-Source Photometric Stereo , 2008, ECCV.

[64]  Jean-Denis Durou,et al.  Numerical methods for shape-from-shading: A new survey with benchmarks , 2008, Comput. Vis. Image Underst..

[65]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[66]  R. Newcomb VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS , 2010 .