Separating Real and Virtual Objects from Their Overlapping Images

We often see scenes where an object's virtual image is reflected on window glass and overlaps with an image of another object behind the glass. This paper proposes a method for separating real and virtual objects from the overlapping images. Our method is based on the optical property that light reflected on glass is polarized, while light transmitted through glass is less polarized. It is possible to eliminate reflected light with a polarizing filter. The polarization direction, however, changes even for planar glass and is not easily determined without information about the position and orientation of the glass and objects relative to the camera. Our method uses a series of images obtained by rotating a polarizing filter. Real objects are separated by selecting the minimum image intensity among a series of images for each pixel. The virtual image of objects is obtained by subtracting the image of the real objects from the image having the maximum image intensity among a series of images for each pixel. We present experiments with actual scenes to demonstrate the effectiveness of the proposed method.