A Dataset of Flash and Ambient Illumination Pairs from the Crowd

Illumination is a critical element of photography and is essential for many computer vision tasks. Flash light is unique in the sense that it is a widely available tool for easily manipulating the scene illumination. We present a dataset of thousands of ambient and flash illumination pairs to enable studying flash photography and other applications that can benefit from having separate illuminations. Different than the typical use of crowdsourcing in generating computer vision datasets, we make use of the crowd to directly take the photographs that make up our dataset. As a result, our dataset covers a wide variety of scenes captured by many casual photographers. We detail the advantages and challenges of our approach to crowdsourcing as well as the computational effort to generate completely separate flash illuminations from the ambient light in an uncontrolled setup. We present a brief examination of illumination decomposition, a challenging and underconstrained problem in flash photography, to demonstrate the use of our dataset in a data-driven approach.

[1]  Luc Van Gool,et al.  DARN: a Deep Adversial Residual Network for Intrinsic Image Decomposition , 2016, ArXiv.

[2]  R. Fergus,et al.  Dark flash photography , 2009, ACM Trans. Graph..

[3]  Balazs Kovacs,et al.  Shading Annotations in the Wild , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Frédo Durand,et al.  Computational bounce flash for indoor portraits , 2016, ACM Trans. Graph..

[5]  Harry Shum,et al.  Flash Cut: Foreground Extraction with Flash and No-flash Image Pairs , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  R. Kouskouridas,et al.  Improving the robustness in feature detection by local contrast enhancement , 2012, 2012 IEEE International Conference on Imaging Systems and Techniques Proceedings.

[8]  Christian Theobalt,et al.  Live intrinsic video , 2016, ACM Trans. Graph..

[9]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[10]  Marc Levoy,et al.  The Frankencamera: an experimental platform for computational photography , 2010, SIGGRAPH 2010.

[11]  Jiansheng Chen,et al.  Face Image Relighting using Locally Constrained Global Optimization , 2010, ECCV.

[12]  Dong Guo,et al.  Robust flash deblurring , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Ramesh Raskar,et al.  Removing photography artifacts using gradient projection and flash-exposure sampling , 2005, SIGGRAPH 2005.

[14]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[15]  Rynson W. H. Lau,et al.  Saliency Detection with Flash and No-flash Image Pairs , 2014, ECCV.

[16]  Michael F. Cohen,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[17]  Hans-Peter Seidel,et al.  Animating deformable objects using sparse spacetime constraints , 2014, ACM Trans. Graph..

[18]  Balazs Kovacs,et al.  Intrinsic Decompositions for Image Editing , 2017, Comput. Graph. Forum.

[19]  Frédo Durand,et al.  Flash photography enhancement via intrinsic relighting , 2004, SIGGRAPH 2004.

[20]  Harry Shum,et al.  Flash matting , 2006, ACM Trans. Graph..

[21]  Wojciech Matusik,et al.  Crowd-Guided Ensembles: How Can We Choreograph Crowd Workers for Video Segmentation? , 2018, CHI.

[22]  Noah Snavely,et al.  Intrinsic images in the wild , 2014, ACM Trans. Graph..

[23]  Pieter Peers,et al.  Post-production facial performance relighting using reflectance transfer , 2007, SIGGRAPH 2007.

[24]  Aswin C. Sankaranarayanan,et al.  Illuminant Spectra-Based Source Separation Using Flash Photography , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[25]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[26]  Manuel M. Oliveira,et al.  Domain transform for edge-aware image and video processing , 2011, SIGGRAPH 2011.

[27]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[28]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[29]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[30]  Bernhard Schölkopf,et al.  EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[31]  Aswin C. Sankaranarayanan,et al.  White balance under mixed illumination using flash photography , 2016, 2016 IEEE International Conference on Computational Photography (ICCP).

[32]  M. Gross,et al.  Analysis of human faces using a measurement-based skin reflectance model , 2006, ACM Trans. Graph..

[33]  G. Schweighofer.  Groupwise Geometric and Photometric Direct Image Registration , 2006 .

[34]  Shree K. Nayar,et al.  All the Images of an Outdoor Scene , 2002, ECCV.

[35]  Kari Pulli,et al.  Robust stereo with flash and no-flash image pairs , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.