Efficient light transport acquisition by coded illumination and robust photometric stereo by dual photography using deep neural network

We present an efficient and robust photometric stereo (PS) measurement by a setup with an optical diffuser. The setup which includes a single projector that places point light sources on the diffuser, extends the possibility of flexible measurement without the limitation from employing physical light sources, i.e. a number and illumination shape. By taking advantage of the setup, we design an illumination for effective and robust surface normal estimation. Unlike the previous techniques, we utilize deep neural network consists of a renderer and a PS module to find multiplexed illumination patterns, which are suitable for PS measurement. Another challenging problem is to measure objects with micro-structures which reflect the light randomly according to lighting and viewing directions. To overcome the problem, we propose a novel PS measurement using a dual-photography setup, which allows us to analyze the angular distribution of reflection by capturing reflection pattern on the diffuser. We show a smooth surface normal can be estimated by simply applying a low-pass filter on the captured images. Moreover, we also propose an effective sampling to deal with time-consuming measurement of dual photography setup. We show that by utilization of the trained sampling codes by DNN considering light transport in the setup, the number of the measurement is drastically reduced.