Blur robust optical flow using motion channel

Abstract It is hard to estimate optical flow given a realworld video sequence with camera shake and other motion blur. In this paper, we first investigate the blur parameterisation for video footage using near linear motion elements. We then combine a commercial 3D pose sensor with an RGB camera, in order to film video footage of interest together with the camera motion. We illustrate that this additional camera motion/trajectory channel can be embedded into a hybrid framework by interleaving an iterative blind deconvolution and warping based optical flow scheme. Our method yields improved accuracy within three other state-of-the-art baselines given our proposed ground truth blurry sequences; and several other realworld sequences filmed by our imaging system.

[1]  Li Xu,et al.  Unnatural L0 Sparse Representation for Natural Image Deblurring , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Wenbin Li,et al.  Video interpolation using optical flow and Laplacian smoothness , 2016, Neurocomputing.

[3]  Jiaya Jia,et al.  High-quality motion deblurring from a single image , 2008, ACM Trans. Graph..

[4]  Sylvain Paris,et al.  Handling Noise in Single Image Deblurring Using Directional Filters , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Qiang Wu,et al.  Directional high-pass filter for blurry image analysis , 2012, Signal Process. Image Commun..

[6]  Judy O'Rourke.  Game on. , 2010, Rehab management.

[7]  Zhihan Lv,et al.  Game On, Science - How Video Game Technology May Help Biologists Tackle Visualization Challenges , 2013, PloS one.

[8]  Ce Liu,et al.  Exploring new representations and applications for motion analysis , 2009 .

[9]  Julian Padget,et al.  Decoupling Cognitive Agents and Virtual Environments , 2012, CAVE.

[10]  Wenbin Li,et al.  Optical Flow Estimation Using Laplacian Mesh Energy , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Wenbin Li.  Nonrigid surface tracking, analysis and evaluation , 2013 .

[12]  Wenbin Li,et al.  Dense Motion Estimation for Smoke , 2016, ACCV.

[13]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[14]  Wenbin Li,et al.  Roto++ , 2016, ACM Trans. Graph..

[15]  S. B. Kang,et al.  Image deblurring using inertial measurement sensors , 2010, ACM Trans. Graph..

[16]  Li Zhang,et al.  Optical flow in the presence of spatially-varying motion blur , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Ioannis M. Rekleitis,et al.  Optical Flow from Motion Blurred Color Images , 2009, 2009 Canadian Conference on Computer and Robot Vision.

[18]  Zhihan Lv,et al.  Multimodal Hand and Foot Gesture Interaction for Handheld Devices , 2014, TOMM.

[19]  Michael J. Black,et al.  Lessons and Insights from Creating a Synthetic Optical Flow Benchmark , 2012, ECCV Workshops.

[20]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[21]  Wenbin Li,et al.  Nonrigid Optical Flow Ground Truth for Real-World Scenes With Time-Varying Shading Effects , 2017, IEEE Robotics and Automation Letters.

[22]  Frédo Durand,et al.  Motion-invariant photography , 2008, ACM Trans. Graph..

[23]  Peter Hedman,et al.  Multi-view Reconstruction of Highly Specular Surfaces in Uncontrolled Environments , 2015, 2015 International Conference on 3D Vision.

[24]  D. Cosker,et al.  Global Alignment for Dynamic 3 D Morphable Model Construction , 2012 .

[25]  Jian Sun,et al.  Progressive inter-scale and intra-scale non-blind image deconvolution , 2008, SIGGRAPH 2008.

[26]  Stephen Lin,et al.  Motion-aware noise filtering for deblurring of noisy and blurry images , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[28]  Gang Ren,et al.  Robust optical flow estimation for continuous blurred scenes using RGB-motion imaging and directional filtering , 2014, IEEE Winter Conference on Applications of Computer Vision.

[29]  Seungyong Lee,et al.  Fast motion deblurring , 2009, ACM Trans. Graph..

[30]  Wenbin Li,et al.  An Anchor Patch Based Optimization Framework for Reducing Optical Flow Drift in Long Image Sequences , 2012, ACCV.

[31]  Dani Lischinski,et al.  Non-rigid dense correspondence with applications for image enhancement , 2011, ACM Trans. Graph..

[32]  Michael J. Black,et al.  A Naturalistic Open Source Movie for Optical Flow Evaluation , 2012, ECCV.

[33]  Wenbin Li,et al.  Drift robust non-rigid optical flow enhancement for long sequences , 2016, J. Intell. Fuzzy Syst..

[34]  Y Matsushita,et al.  An Anchor Patch Based Optimization Framework For Reducing Optical Flow Drift in Long Image Sequences , 2012 .

[35]  Stephen Lin,et al.  Image/video deblurring using a hybrid camera , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.