Relighting of Dynamic Video

We present a novel method to relight video sequences for multimedia applications given known surface shape and original illumination. The method preserves fine visual details. It requires single view video frames, approximate 3D shape and known illumination only, making it applicable for multimedia and studio production. The technique is demonstrated for relighting video sequences of faces.

[1]  Dimitris N. Metaxas,et al.  Incorporating Illumination Constraints in Deformable Models for Shape from Shading and Light Direction Estimation , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Katsushi Ikeuchi,et al.  Separating reflection components based on chromaticity and noise analysis , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Jitendra Malik,et al.  Recovering photometric properties of architectural scenes from photographs , 1998, SIGGRAPH.

[4]  Adrian Hilton,et al.  Video-rate capture of dynamic face shape and appearance , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[5]  Andrew Gardner,et al.  Animatable Facial Reflectance Fields , 2004 .

[6]  H C Lee,et al.  Method for computing the scene-illuminant chromaticity from specular highlights. , 1986, Journal of the Optical Society of America. A, Optics and image science.

[7]  Paul Debevec,et al.  Inverse global illumination: Recovering re?ectance models of real scenes from photographs , 1998 .

[8]  Takeo Kanade,et al.  Surface Reflection: Physical and Geometrical Perspectives , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  George Drettakis,et al.  Interactive Virtual Relighting of Real Scenes , 2000, IEEE Trans. Vis. Comput. Graph..

[10]  Katsushi Ikeuchi,et al.  Separating reflection components of textured surfaces using a single image , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Katsushi Ikeuchi,et al.  Object shape and reflectance modeling from observation , 1997, SIGGRAPH.

[12]  Adrian Hilton,et al.  Relighting of Facial Images , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[13]  Steven M. Seitz,et al.  Example-based photometric stereo: shape reconstruction with general, varying BRDFs , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Ronen Basri,et al.  Lambertian reflectance and linear subspaces , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[15]  Donald P. Greenberg,et al.  A comprehensive physical model for light reflection , 1991, SIGGRAPH.

[16]  Andrew Gardner,et al.  Performance relighting and reflectance transformation with time-multiplexed illumination , 2005, ACM Trans. Graph..

[17]  Shoji Tominaga,et al.  Surface Identification Using the Dichromatic Reflection Model , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Claus B. Madsen,et al.  A Model-Based Approach to Image Relighting with a Potential for Real-Time Implementation , 2005, VVG.