Automatic Reflection Removal using Gradient Intensity and Motion Cues

We present a method to separate the background image and reflection from two photos that are taken in front of a transparent glass under slightly different viewpoints. In our method, the SIFT-flow between two images is first calculated and a motion hierarchy is constructed from the SIFT-flow at multiple levels of spatial smoothness. To distinguish background edges and reflection edges, we calculate a motion score for each edge pixel by its variance along the motion hierarchy. Alternatively, we make use of the so-called superpixels to group edge pixels into edge segments and calculate the motion scores by averaging over each segment. In the meantime, we also calculate an intensity score for each edge pixel by its gradient magnitude. We combine both motion and intensity scores to get a combination score. A binary labelling (for separation) can be obtained by thresholding the combination scores. The background image is finally reconstructed from the separated gradients. Compared to the existing approaches that require a sequence of images or a video clip for the separation, we only need two images, which largely improves its feasibility. Various challenging examples are tested to validate the effectiveness of our method.