Estimation of Environmental Lighting from Known Geometries for Mobile Augmented Reality

Light source estimation and virtual lighting must be believable in terms of appearance and correctness in augmented reality scenes. As a result of illumination complexity in an outdoor scene, realistic lighting for augmented reality is still a challenging problem. In this paper, we propose a framework based on an estimation of environmental lighting from well-defined objects, specifically human faces. The method is tuned for outdoor use, and the algorithm is further enhanced to illuminate virtual objects exposed to direct sunlight. Our model can be integrated into existing mobile augmented reality frameworks to enhance visual perception.

[1]  Peter-Pike J. Sloan,et al.  Clustered principal components for precomputed radiance transfer , 2003, ACM Trans. Graph..

[2]  Jan Kautz,et al.  Precomputed radiance transfer for real-time rendering in dynamic, low-frequency lighting environments , 2002 .

[3]  Berthold K. P. Horn.  Obtaining shape from shading information , 1989 .

[4]  Norimichi Tsumura,et al.  Estimating the Directions to Light Sources Using Images of Eye for Reconstructing 3D Human Face , 2003, CIC.

[5]  Naokazu Yokoya,et al.  Geometric and photometric registration for real-time augmented reality , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[6]  James M. Van Verth,et al.  Essential Mathematics for Games and Interactive Applications , 2015 .

[7]  Stephen Lin,et al.  Separating reflections in human iris images for illumination estimation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[8]  Alex Pentland.  Linear shape from shading , 2004, International Journal of Computer Vision.

[9]  Michael J. Brooks,et al.  The variational approach to shape from shading , 1986, Comput. Vis. Graph. Image Process..

[10]  Naokazu Yokoya,et al.  Real-time estimation of light source environment for photorealistic augmented reality , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[11]  Shree K. Nayar,et al.  Eyes for relighting , 2004, SIGGRAPH 2004.

[12]  Erik Reinhard,et al.  Image-based material editing , 2005, SIGGRAPH '05.

[13]  Claus B. Madsen,et al.  Real-time image based lighting for outdoor augmented reality under dynamically changing illumination conditions , 2006, GRAPP.

[14]  Ronen Basri,et al.  Lambertian Reflectance and Linear Subspaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[16]  Erik Reinhard,et al.  Image-based material editing , 2005, SIGGRAPH '05.

[17]  Michael Wimmer,et al.  Efficient Spherical Harmonics Lighting with the Preetham Skylight Model , 2008, Eurographics.

[18]  Michael J. Brooks,et al.  Shape and Source from Shading , 1985, IJCAI.

[19]  Baback Moghaddam,et al.  A Practical Face Relighting Method for Directional Lighting Normalization , 2005, AMFG.

[20]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[21]  Peter Shirley,et al.  A practical analytic model for daylight , 1999, SIGGRAPH.

[22]  J. Edward Swan,et al.  Survey of User-Based Experimentation in Augmented Reality , 2005 .

[23]  Shoji Tominaga,et al.  Spectral Measurement of Ambient Lighting and Its Application to Image Rendering , 2002, MVA.

[24]  Steven K. Feiner,et al.  Computer graphics: principles and practice (2nd ed.) , 1990 .

[25]  E. Reinhard.  Photographic Tone Reproduction for Digital Images , 2002 .

[26]  D N Chandrappa,et al.  Face Detection Using a Boosted Cascade of Features Using OpenCV , 2012 .

[27]  Katsushi Ikeuchi,et al.  Illumination distribution from shadows , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[28]  Paul Debevec.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 2008, SIGGRAPH Classes.

[29]  Erik Reinhard,et al.  Light Source Detection in Photographs , 2009, CEIG.

[30]  Robin Green,et al.  Spherical Harmonic Lighting: The Gritty Details , 2003 .

[31]  Josef Kittler,et al.  Robust albedo estimation from face image under unknown illumination , 2008, SPIE Defense + Commercial Sensing.