Height-from-Polarisation with Unknown Lighting or Albedo

We present a method for estimating surface height directly from a single polarisation image simply by solving a large, sparse system of linear equations. To do so, we show how to express polarisation constraints as equations that are linear in the unknown height. The local ambiguity in the surface normal azimuth angle is resolved globally when the optimal surface height is reconstructed. Our method is applicable to dielectric objects exhibiting diffuse and specular reflectance, though lighting and albedo must be known. We relax this requirement by showing that either spatially varying albedo or illumination can be estimated from the polarisation image alone using nonlinear methods. In the case of illumination, the estimate can only be made up to a binary ambiguity which we show is a generalised Bas-relief transformation corresponding to the convex/concave ambiguity. We believe that our method is the first passive, monocular shape-from-x technique that enables well-posed height estimation with only a single, uncalibrated illumination condition. We present results on real world data, including in uncontrolled, outdoor illumination.

[1]  Fabrice Meriaudeau,et al.  Polarization imaging applied to 3D reconstruction of specular metallic surfaces , 2005, IS&T/SPIE Electronic Imaging.

[2]  M. Krejsa,et al.  Structural Mechanics , 2001 .

[3]  Rin-ichiro Taniguchi,et al.  Shape and light directions from shading and polarization , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Valery V. Tuchin,et al.  Optical polarization in biomedical applications , 2006 .

[5]  Katsushi Ikeuchi,et al.  Transparent surface modeling from a pair of polarization images , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Cong Phuoc Huynh,et al.  Shape and Refractive Index from Single-View Spectro-Polarimetric Images , 2012, International Journal of Computer Vision.

[7]  Jan Kautz,et al.  Polarimetric Multi-view Stereo , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Edwin R. Hancock,et al.  Shape Estimation Using Polarization and Shading from Two Views , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Cong Phuoc Huynh,et al.  Shape and refractive index recovery from single-view polarisation images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  L. B. Wolff Diffuse-reflectance model for smooth dielectric surfaces , 1994 .

[11]  Stephen P. Boyd,et al.  Disciplined Convex Programming , 2006 .

[12]  Lawrence B. Wolff,et al.  Polarization vision: a new sensory approach to image understanding , 1997, Image Vis. Comput..

[13]  Adrien Bartoli,et al.  Shape-from-Polarization in laparoscopy , 2013, 2013 IEEE 10th International Symposium on Biomedical Imaging.

[14]  William A. P. Smith,et al.  Linear Depth Estimation from an Uncalibrated, Monocular Polarisation Image , 2016, ECCV.

[15]  Gary A. Atkinson,et al.  Polarisation photometric stereo , 2017, Comput. Vis. Image Underst..

[16]  Szymon Rusinkiewicz,et al.  Efficiently combining positions and normals for precise 3D geometry , 2005, ACM Trans. Graph..

[17]  David J. Kriegman,et al.  The Bas-Relief Ambiguity , 2004, International Journal of Computer Vision.

[18]  Ramesh Raskar,et al.  Polarized 3D: High-Quality Depth Sensing with Polarization Cues , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[19]  Alessio Del Bue,et al.  Direct Differential Photometric Stereo Shape Recovery of Diffuse and Specular Surfaces , 2016, Journal of Mathematical Imaging and Vision.

[20]  O. Drbohlav,et al.  Unambiguous determination of shape from photometric stereo with unknown light sources , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[21]  Masashi Baba,et al.  Surface normal estimation of black specular objects from multiview polarization images , 2016 .

[22]  Aly A. Farag,et al.  Direct method for shape recovery from polarization and shading , 2012, 2012 19th IEEE International Conference on Image Processing.

[23]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[24]  W. Marsden I and J , 2012 .

[25]  Paul E. Debevec,et al.  Multiview face capture using polarized spherical gradient illumination , 2011, ACM Trans. Graph..

[26]  Diego F. Nehab,et al.  Efficiently combining positions and normals for precise 3D geometry , 2005, SIGGRAPH 2005.

[27]  Pascal Getreuer,et al.  Total Variation Inpainting using Split Bregman , 2012, Image Process. Line.

[28]  J. Kautz,et al.  Polarimetric MultiView Stereo , 2017 .

[29]  Jannik Boll Nielsen,et al.  On optimal, minimal BRDF sampling for reflectance acquisition , 2015, ACM Trans. Graph..

[30]  Stefan Rahmann,et al.  Reconstruction of specular surfaces using polarization imaging , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[31]  James F. Blinn,et al.  Models of light reflection for computer synthesized pictures , 1977, SIGGRAPH.

[32]  Gary A. Atkinson,et al.  Recovery of surface orientation from diffuse polarization , 2006, IEEE Transactions on Image Processing.

[33]  Terrance E. Boult,et al.  Constraining Object Features Using a Polarization Reflectance Model , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Katsushi Ikeuchi,et al.  Polarization-based inverse rendering from a single view , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[35]  Edwin R. Hancock,et al.  Surface Reconstruction Using Polarization and Photometric Stereo , 2007, CAIP.

[36]  Steven A. Shafer,et al.  Using color to separate reflection components , 1985 .

[37]  Cong Phuoc Huynh,et al.  Imaging Spectroscopy for Scene Analysis , 2012, Advances in Computer Vision and Pattern Recognition.

[38]  Yoav Y. Schechner Self-Calibrating Imaging Polarimetry , 2015, 2015 IEEE International Conference on Computational Photography (ICCP).

[39]  Masashi Baba,et al.  Polarization-Based Surface Normal Estimation of Black Specular Objects from Multiple Viewpoints , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[40]  Ramesh Raskar,et al.  Depth Sensing Using Geometrically Constrained Polarization Normals , 2017, International Journal of Computer Vision.