Generating high quality pan-shots from motion blurred videos

Abstract In this paper, we demonstrate a method to generate a pan photo from a video captured using a hand-held camera. We handle three relevant scenarios (no blur, only foreground blurred, both foreground and background blurred) that arise while capturing a video with a fixed frame-rate and under different relative object velocities and camera motion. Our method first segments out the moving object by motion compensation of the background and then estimates the inter-frame velocity and relative depth of the object. Automatic gradient-based identification is then performed to classify the video into one of the above three scenarios. If only the foreground is blurred, we perform non-blind restoration by judiciously harnessing the background motion and foreground velocity to estimate the foreground blur. When the background is blurred too, a blind multi-frame deblurring approach is advocated to get the background motion which is used to infer the foreground blur to obtain the latent frames. Once clean frames are obtained, we align the object and rewarp the background with respect to the net displacement of the object in each frame which when averaged produces the required realistic pan-photo. We demonstrate our method on a number of videos captured using different consumer cameras as well as on videos downloaded from the Internet.