Shape from Specular Reflection and Optical Flow

Abstract Inferring scene geometry from a sequence of camera images is one of the central problems in computer vision. While the overwhelming majority of related research focuses on diffuse surface models, there are cases when this is not a viable assumption: in many industrial applications, one has to deal with metal or coated surfaces exhibiting a strong specular behavior. We propose a novel and generalized constrained gradient descent method to determine the shape of a purely specular object from the reflection of a calibrated scene and additional data required to find a unique solution. This data is exemplarily provided by optical flow measurements obtained by small scale motion of the specular object, with camera and scene remaining stationary. We present a non-approximative general forward model to predict the optical flow of specular surfaces, covering rigid body motion as well as elastic deformation, and allowing for a characterization of problematic points. We demonstrate the applicability of our method by numerical experiments on synthetic and real data.

[1]  Henrik Aanæs,et al.  A variational analysis of shape from specularities using sparse data , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[2]  Michael Lindenbaum,et al.  Dense Mirroring Surface Recovery from 1D Homographies and Sparse Correspondences , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Niels Chr. Overgaard,et al.  A Geometric Formulation of Gradient Descent for Variational Problems with Moving Surfaces , 2005, Scale-Space.

[4]  Min Chen,et al.  Theory and application of specular path perturbation , 2000, TOGS.

[5]  Stefano Soatto,et al.  Estimation of 3D Surface Shape and Smooth Radiance from 2D Images: A Level Set Approach , 2003, J. Sci. Comput..

[6]  Shree K. Nayar,et al.  A Theory of Specular Surface Geometry , 2004, International Journal of Computer Vision.

[7]  Michael Lindenbaum,et al.  Dense mirroring surface recovery from 1D homographies and sparse correspondences , 2007, CVPR.

[8]  O. Faugeras,et al.  A VISCOSITY SOLUTION METHOD FOR SHAPE-FROM-SHADING WITHOUT IMAGE BOUNDARY DATA , 2006 .

[9]  Olivier D. Faugeras,et al.  Designing spatially coherent minimizing flows for variational problems based on active contours , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[10]  R. Andrew Hicks,et al.  The method of vector fields for catadioptric sensor design with applications to panoramic imaging , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[11]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[12]  Ronald Fedkiw,et al.  Level set methods and dynamic implicit surfaces , 2002, Applied mathematical sciences.

[13]  Peter F. Sturm,et al.  Voxel carving for specular surfaces , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[14]  Y. J. Tejwani,et al.  Robot vision , 1989, IEEE International Symposium on Circuits and Systems,.

[15]  Berthold K. P. Horn.  SHAPE FROM SHADING: A METHOD FOR OBTAINING THE SHAPE OF A SMOOTH OPAQUE OBJECT FROM ONE VIEW , 1970 .

[16]  M. Burger.  A framework for the construction of level set methods for shape optimization and reconstruction , 2003 .

[17]  V. Rovenski,et al.  Differential Geometry of Curves and Surfaces , 1952, Nature.

[18]  Marcus A. Magnor,et al.  Weighted Minimal Hypersurfaces and Their Applications in Computer Vision , 2004, ECCV.

[19]  Sören Kammel,et al.  Deflektometrische Untersuchung spiegelnd reflektierender Freiformflächen , 2005 .

[20]  Michael J. Black,et al.  Specular Flow and the Recovery of Surface Structure , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[21]  Jonathan Balzer,et al.  Regularization of the deflectometry problem using shading data , 2006, SPIE Optics East.

[22]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[23]  Andrew Blake,et al.  The information available to a moving observer from specularities , 1989, Image Vis. Comput..

[24]  Olivier D. Faugeras,et al.  Image Segmentation Using Active Contours: Calculus of Variations or Shape Gradients? , 2003, SIAM J. Appl. Math..

[25]  Thomas Bonfort.  Specular Surface Reconstruction from Images. (Reconstruction de Surfaces Réfléchissantes à partir d'Images) , 2003 .

[26]  Michel Barlaud,et al.  Shape Gradient for Image and Video Segmentation , 2006, Handbook of Mathematical Models in Computer Vision.

[27]  Matthew MacDonald,et al.  Shapes and Geometries , 1987 .

[28]  Niels Chr. Overgaard,et al.  A Gradient Descent Procedure for Variational Dynamic Surface Problems with Constraints , 2005, VLSM.

[29]  Min Chen,et al.  Local Shape from Mirror Reflections , 2005, International Journal of Computer Vision.

[30]  Jan Mayer.  Symmetric Permutations for I-matrices to Delay and Avoid Small Pivots During Factorization , 2008, SIAM J. Sci. Comput..

[31]  Manfredo P. do Carmo,et al.  Differential geometry of curves and surfaces , 1976 .

[32]  James A. Sethian,et al.  Level Set Methods and Fast Marching Methods , 1999 .

[33]  Nikos Paragios,et al.  Handbook of Mathematical Models in Computer Vision , 2005 .

[34]  Brian A. Barsky,et al.  Reconstructing curved surfaces from specular reflection patterns using spline surface fitting of normals , 1996, SIGGRAPH.