Space Relighter: Recording and Reproducing Illumination in Real Scene

We introduce SpaceRelighter , a projector-based mixed reality system that records illumination in a real scene and reproduces it at a later time by projecting a light pattern onto the scene. Users can see a virtually illuminated real scene without having to wear special devices such as a head-mounted display. Because the virtual illumination and real scene are merged in a real 3-D environment and not on a 2-D display, users have a better sense of reality. We first explain the principle of estimating light patterns by photometric and geometric conversion. We then describe an extended method for reducing the problem of occlusion by using multiple cameras and multiple projectors. We constructed a prototype system including two cameras and two projectors to confirm that illumination can be reproduced using projectors.