Training many-parameter shape-from-shading models using a surface database

Shape-from-shading (SFS) methods tend to rely on models with few parameters because these parameters need to be hand-tuned. This limits the number of different cues that the SFS problem can exploit. In this paper, we show how machine learning can be applied to an SFS model with a large number of parameters. Our system learns a set of weighting parameters that use the intensity of each pixel in the image to gauge the importance of that pixel in the shape reconstruction process. We show empirically that this leads to a significant increase in the accuracy of the recovered surfaces. Our learning approach is novel in that the parameters are optimized with respect to actual surface output by the system. In the first, offline phase, a hemisphere is rendered using a known illumination direction. The isophotes in the resulting reflectance map are then modelled using Gaussian mixtures to obtain a parametric representation of the isophotes. This Gaussian parameterization is then used in the second phase to learn intensity-based weights using a database of 3D shapes. The weights can also be optimized for a particular input image.

[1]  William T. Freeman,et al.  Learning Low-Level Vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[2]  Gerd Hirzinger,et al.  Learning shape from shading by a multilayer network , 1996, IEEE Trans. Neural Networks.

[3]  David C. Knill,et al.  Learning a Near-Optimal Estimator for Surface Shape from Shading , 1990, Comput. Vis. Graph. Image Process..

[4]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.

[5]  Jezekiel Ben-Arie,et al.  A neural network approach for reconstructing surface shape from shading , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[6]  Prateek Jain,et al.  Fast image search for learned metrics , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Olivier D. Faugeras,et al.  A Unifying and Rigorous Shape from Shading Method Adapted to Realistic Data and Applications , 2006, Journal of Mathematical Imaging and Vision.

[8]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[9]  David J. Evans,et al.  A Neural Network Approach To Shape From Shading , 2003, Int. J. Comput. Math..

[10]  Jean-Denis Durou,et al.  Numerical methods for shape-from-shading: A new survey with benchmarks , 2008, Comput. Vis. Image Underst..

[11]  Ruigang Yang,et al.  Examplar-based Shape from Shading , 2007, Sixth International Conference on 3-D Digital Imaging and Modeling (3DIM 2007).

[12]  Edwin R. Hancock,et al.  Facial Shape-from-shading and Recognition Using Principal Geodesic Analysis and Robust Statistics , 2007, International Journal of Computer Vision.

[13]  Mubarak Shah,et al.  Shape from shading using linear approximation , 1994, Image Vis. Comput..

[14]  Tommy W. S. Chow,et al.  Neural computation approach for developing a 3D shape reconstruction model , 2001, IEEE Trans. Neural Networks.

[15]  Brian Potetz,et al.  Efficient Belief Propagation for Vision Using Linear Constraint Nodes , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Edwin R. Hancock,et al.  New Constraints on Data-Closeness and Needle Map Consistency for Shape-from-Shading , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Terrence J. Sejnowski,et al.  Network model of shape-from-shading: neural function arises from both receptive and projective fields , 1988, Nature.

[19]  Marshall F. Tappen,et al.  Utilizing Variational Optimization to Learn Markov Random Fields , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Christoph H. Lampert,et al.  Learning to Localize Objects with Structured Output Regression , 2008, ECCV.

[21]  Alex Pentland.  Linear shape from shading , 2004, International Journal of Computer Vision.

[22]  Michael J. Brooks,et al.  The variational approach to shape from shading , 1986, Comput. Vis. Graph. Image Process..

[23]  Richard C. Wilson,et al.  Belief Propagation with Directional Statistics for Solving the Shape-from-Shading Problem , 2008, ECCV.

[24]  Michael J. Black,et al.  Fields of Experts: a framework for learning image priors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).