Light-Field Intrinsic Dataset

Light-field imaging has various advantages over the traditional 2D photography, such as depth estimation and occlusion detection, which can aid intrinsic decomposition. The extracted intrinsic layers enable multiple applications, such as light-field appearance editing. However, the current light-field intrinsic decomposition techniques primarily resort to qualitative comparisons, due to lack of ground-truth data. In this work, we address this problem by providing intrinsic dataset for real world and synthetic 4D and 3D (only horizontal parallax) light fields. The ground-truth intrinsic data comprises albedo, shading and specularity layers for all sub-aperture images. In case of synthetic data, we also provide ground-truth depth, normals, and further decomposition of shading into direct and indirect components. For real-world data acquisition, we make use of custom hardware and 3D printed objects, assuring precision during multi-pass capturing. We also perform, qualitative and quantitative, comparison of existing intrinsic decomposition algorithms for single image, video, and light field. To the best of our knowledge, this is the first such dataset for light fields, which is also applicable for single image, multi-view stereo, and video. c © 2018. The copyright of this document resides with its authors. It may be distributed unchanged freely in print or electronic forms. *indicates equal contribution

[1]  Jitendra Malik,et al.  Shape, Illumination, and Reflectance from Shading , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Abhijeet Ghosh,et al.  Single-shot Layered Reflectance Separation Using a Polarized Light Field Camera , 2016, EGSR.

[3]  Edwin Herbert Land,et al.  The Retinex Theory of Color Vision SCIENTIFIC AMERICAN , 2009 .

[4]  Qionghai Dai,et al.  Intrinsic video and applications , 2014, ACM Trans. Graph..

[5]  Stephen Lin,et al.  A Closed-Form Solution to Retinex with Nonlocal Texture Constraints , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Adrien Bousseau,et al.  Coherent intrinsic images from photo collections , 2012, ACM Trans. Graph..

[7]  Hans-Peter Seidel,et al.  Light-Field Appearance Editing based on Intrinsic Decomposition , 2018, Journal of Perceptual Imaging.

[8]  H. Barrow,et al.  RECOVERING INTRINSIC SCENE CHARACTERISTICS FROM IMAGES , 1978 .

[9]  Ko Nishino,et al.  Radiometric Scene Decomposition: Scene Reflectance, Illumination, and Geometry from RGB-D Images , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[10]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[11]  Anna Alperovich,et al.  Reflection Separation in Light Fields based on Sparse Coding and Specular Flow , 2016, VMV.

[12]  Christian Theobalt,et al.  Live intrinsic video , 2016, ACM Trans. Graph..

[13]  Noah Snavely,et al.  Intrinsic images in the wild , 2014, ACM Trans. Graph..

[14]  Bernard Ghanem,et al.  Intrinsic Scene Decomposition from RGB-D Images , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Bastian Goldlücke,et al.  Light Field Intrinsics with a Deep Encoder-Decoder Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[16]  Bing Zeng,et al.  Intrinsic decomposition for stereoscopic images , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[17]  Dmitry Chetverikov,et al.  A Survey of Specularity Removal Methods , 2011, Comput. Graph. Forum.

[18]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[19]  Adrien Bousseau,et al.  Multiview Intrinsic Images of Outdoors Scenes with an Application to Relighting , 2015, ACM Trans. Graph..

[20]  Stella X. Yu,et al.  Direct Intrinsics: Learning Albedo-Shading Decomposition by Convolutional Regression , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[21]  Hans-Peter Seidel,et al.  Towards a Quality Metric for Dense Light Fields , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Daniel Cremers,et al.  Dense Multi-view 3D-reconstruction Without Dense Correspondences , 2017, ArXiv.

[23]  Paul Graham,et al.  Acquiring reflectance and shape from continuous spherical harmonic illumination , 2013, ACM Trans. Graph..

[24]  Theo Gevers,et al.  CNN Based Learning Using Reflection and Retinex Models for Intrinsic Image Decomposition , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[25]  Cedric Nishan Canagarajah,et al.  Structural Similarity-Based Object Tracking in Video Sequences , 2006, 2006 9th International Conference on Information Fusion.

[26]  Xiaoyue Jiang,et al.  Intrinsic Image Decomposition: A Comprehensive Review , 2017, ICIG.

[27]  Pierre-Yves Laffont,et al.  Intrinsic Decomposition of Image Sequences from Local Temporal Variations , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[28]  Andreas Kolb,et al.  Multi-view Multi-illuminant Intrinsic Dataset , 2016, BMVC.

[29]  Ko Nishino,et al.  Shape and Reflectance Estimation in the Wild , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Jitendra Malik,et al.  Intrinsic Scene Properties from a Single RGB-D Image , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Stephen Lin,et al.  Shading-Based Shape Refinement of RGB-D Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Michael J. Black,et al.  A Naturalistic Open Source Movie for Optical Flow Evaluation , 2012, ECCV.

[33]  Andreas Kolb,et al.  A Comprehensive Multi-Illuminant Dataset for Benchmarking of the Intrinsic Image Algorithms , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[34]  Bastian Goldlücke,et al.  A Variational Model for Intrinsic Light Field Decomposition , 2016, ACCV.

[35]  Joachim Keinert,et al.  Acquisition system for dense lightfield of large scenes , 2017, 2017 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).

[36]  Bastian Goldlücke,et al.  Shadow and Specularity Priors for Intrinsic Light Field Decomposition , 2017, EMMCVPR.

[37]  Stefan Leutenegger,et al.  ElasticFusion: Real-time dense SLAM and light source estimation , 2016, Int. J. Robotics Res..

[38]  Jiajun Wu,et al.  Self-Supervised Intrinsic Image Decomposition , 2017, NIPS.

[39]  Jitendra Malik,et al.  Color Constancy, Intrinsic Images, and Shape Estimation , 2012, ECCV.

[40]  Balazs Kovacs,et al.  Intrinsic Decompositions for Image Editing , 2017, Comput. Graph. Forum.

[41]  Michael J. Black,et al.  Lessons and Insights from Creating a Synthetic Optical Flow Benchmark , 2012, ECCV Workshops.

[42]  英樹 藤堂,et al.  Interactive intrinsic video editing , 2014, ACM Trans. Graph..

[43]  Marc Levoy,et al.  Using plane + parallax for calibrating dense camera arrays , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[44]  Vladlen Koltun,et al.  A Simple Model for Intrinsic Image Decomposition with Depth Cues , 2013, 2013 IEEE International Conference on Computer Vision.

[45]  Jian Shi,et al.  Learning Non-Lambertian Object Intrinsics Across ShapeNet Categories , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Edward H. Adelson,et al.  Ground truth dataset and baseline evaluations for intrinsic image algorithms , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[47]  Matthias Nießner,et al.  A Lightweight Approach for On-the-Fly Reflectance Estimation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[48]  Kun Zhou,et al.  Intrinsic Light Field Images , 2016, Comput. Graph. Forum.

[49]  Alexei A. Efros,et al.  Learning Data-Driven Reflectance Priors for Intrinsic Image Decomposition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).