Layered RGBD Scene Flow Estimation with Global Non-rigid Local Rigid Assumption

RGBD scene flow has attracted increasing attention in the computer vision community with the popularity of depth sensor. To accurately estimate three-dimensional motion of object, a layered scene flow estimation with global non-rigid, local rigid motion assumption is presented in this paper. Firstly, depth image is inpainted based on RGB image due to original depth image contains noises. Secondly, depth image is layered according to K-means clustering algorithm, which can quickly and simply layer the depth image. Thirdly, scene flow is estimated based on the assumption we proposed. Finally, experiments are implemented on RGBD tracking dataset and deformable 3D reconstruction dataset, and the analysis of quantitative indicators, RMS (Root Mean Square error) and AAE (Average Angular Error). The results show that the proposed method can distinguish moving regions from the static background better, and more accurately estimate the motion information of the scene by comparing with the global rigid, local non-rigid assumption.

[1]  Janis Fehr,et al.  Computing Range Flow from Multi-modal Kinect Data , 2011, ISVC.

[2]  Deqing Sun,et al.  Layered RGBD scene flow estimation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Wei Feng,et al.  Class imbalance ensemble learning based on the margin theory , 2018 .

[4]  Frederic Devernay,et al.  A Variational Method for Scene Flow Estimation from Stereo Sequences , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[5]  Degui Xiao,et al.  Monocular scene flow estimation via variational method , 2015, Multimedia Tools and Applications.

[6]  Zheng Wang,et al.  A deep-learning based feature hybrid framework for spatiotemporal saliency detection inside videos , 2018, Neurocomputing.

[7]  James L. Crowley,et al.  Scene flow by tracking in intensity and depth data , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[8]  Jinchang Ren,et al.  High-Accuracy Sub-Pixel Motion Estimation From Noisy Images in Fourier Domain , 2010, IEEE Transactions on Image Processing.

[9]  Konrad Schindler,et al.  Piecewise Rigid Scene Flow , 2013, 2013 IEEE International Conference on Computer Vision.

[10]  Ping Ma,et al.  A stability constrained adaptive alpha for gravitational search algorithm , 2018, Knowl. Based Syst..

[11]  Xuezhi Xiang,et al.  Scene Flow Estimation Based on 3D Local Rigidity Assumption and Depth Map Driven Anisotropic Smoothness , 2018, IEEE Access.

[12]  Michael J. Black,et al.  Layered image motion with explicit occlusions, temporal consistency, and depth ordering , 2010, NIPS.

[13]  Richard Bowden,et al.  Scene Particles: Unregularized Particle-Based Scene Flow Estimation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Daniel Cremers,et al.  Efficient Dense Scene Flow from Sparse or Dense Stereo Data , 2008, ECCV.

[15]  Yi Zhang,et al.  Depth Inpainting Algorithm of RGB-D Camera Combined with Color Image , 2018, 2018 2nd IEEE Advanced Information Management,Communicates,Electronic and Automation Control Conference (IMCEC).

[16]  Stanley S. Ipson,et al.  Fusion of intensity and inter-component chromatic difference for effective and robust colour edge detection , 2010 .

[17]  Richard Bowden,et al.  Kinecting the dots: Particle based scene flow from depth sensors , 2011, 2011 International Conference on Computer Vision.

[18]  Daniel Cremers,et al.  KillingFusion: Non-rigid 3D Reconstruction without Correspondences , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Feng Wu,et al.  Background Prior-Based Salient Object Detection via Deep Reconstruction Residual , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Takeo Kanade,et al.  Three-dimensional scene flow , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[21]  Jinchang Ren,et al.  Object-Based 2D-to-3D Video Conversion for Effective Stereoscopic Content Generation in 3D-TV Applications , 2011, IEEE Transactions on Broadcasting.

[22]  Michael J. Black,et al.  A Fully-Connected Layered Model of Foreground and Background Flow , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Stephen Marshall,et al.  Cognitive Fusion of Thermal and Visible Imagery for Effective Detection and Tracking of Pedestrians in Videos , 2018, Cognitive Computation.

[24]  Didier Stricker,et al.  SceneFlowFields: Dense Interpolation of Sparse Scene Flow Correspondences , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).