Learning Scene Flow in 3D Point Clouds

Many applications in robotics and human-computer interaction can benefit from an understanding of 3D motion of points in a dynamic environment, widely noted as scene flow. While most previous methods focus on solving the problem with stereo and RGB-D images, few try to estimate scene flow directly from point clouds. In this work, we propose a novel deep neural network named $\textit{FlowNet3D}$ that learns scene flow from point clouds in an end-to-end fashion. Our network simultaneously learns deep hierarchical point cloud features, flow embeddings as well as how to smooth the output. We evaluate the network on both challenging synthetic data and real LiDAR scans from KITTI. Trained on synthetic data only, our network is able to generalize well to real scans. Benefited from learning directly in point clouds, our model achieved significantly more accurate scene flow results compared with various baselines on stereo images and RGB-D images.