CNN-based multi-frame IMO detection from a monocular camera

This paper presents a method for detecting independently moving objects (IMOs) from a monocular camera mounted on a moving car. A CNN-based classifier is employed to generate IMO candidate patches; independent motion is detected by geometric criteria on keypoint trajectories in these patches. Instead of looking only at two consecutive frames, we analyze keypoints inside the IMO candidate patches through multi-frame epipolar consistency checks. The obtained motion labels (IMO/static) are then propagated over time using the combination of motion cues and appearance-based information of the IMO candidate patches. We evaluate the performance of our method on the KITTI dataset, focusing on sub-sequences containing IMOs.

[1]  Martin Jägersand,et al.  MODNet: Moving Object Detection Network with Motion and Appearance for Autonomous Driving , 2017, ArXiv.

[2]  Jongin Lim,et al.  Attention-inspired moving object detection in monocular dashcam videos , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[3]  Daniel Cremers,et al.  Detection and Segmentation of Independently Moving Objects from Dense Scene Flow , 2009, EMMCVPR.

[4]  K. Madhava Krishna,et al.  Realtime moving object detection from a freely moving monocular camera , 2010, 2010 IEEE International Conference on Robotics and Biomimetics.

[5]  Stan Sclaroff,et al.  Learning a Family of Detectors via Multiplicative Kernels , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Thomas Brox,et al.  Variational Motion Segmentation with Level Sets , 2006, ECCV.

[7]  Min Bai,et al.  Exploiting Semantic Information and Deep Matching for Optical Flow , 2016, ECCV.

[8]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[9]  Ezequiel López-Rubio,et al.  Foreground detection for moving cameras with stochastic approximation , 2015, Pattern Recognit. Lett..

[10]  Michael J. Black,et al.  Optical Flow in Mostly Rigid Scenes , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Michael Felsberg,et al.  Good Edgels to Track: Beating the Aperture Problem with Epipolar Geometry , 2014, ECCV Workshops.

[12]  Hugh F. Durrant-Whyte,et al.  Simultaneous Localization, Mapping and Moving Object Tracking , 2007, Int. J. Robotics Res..

[13]  Gaurav S. Sukhatme,et al.  Detecting Moving Objects using a Single Camera on a Mobile Robot in an Outdoor Environment , 2004 .

[14]  Rudolf Mester,et al.  Multimodal scale estimation for monocular visual odometry , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[15]  Takeo Kato,et al.  Vehicle Ego-Motion Estimation and Moving Object Detection using a Monocular Camera , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[16]  A. Jazayeri,et al.  Vehicle Detection and Tracking in Car Video Based on Motion Model , 2011, IEEE Transactions on Intelligent Transportation Systems.

[17]  Gustavo Carneiro,et al.  Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue , 2016, ECCV.

[18]  Rudolf Mester,et al.  Predictive monocular odometry (PMO): What is possible without RANSAC and multiframe bundle adjustment? , 2017, Image Vis. Comput..

[19]  Chunhua Shen,et al.  Depth and surface normal estimation from monocular images using regression on deep features and hierarchical CRFs , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Julius Ziegler,et al.  Sparse scene flow segmentation for moving object detection in urban environments , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[21]  Rudolf Mester,et al.  Motion Priors Estimation for Robust Matching Initialization in Automotive Applications , 2015, ISVC.

[22]  Wolfram Burgard,et al.  Topometric Localization with Deep Learning , 2017, ISRR.

[23]  Rudolf Mester,et al.  Instance-Level Segmentation of Vehicles by Deep Contours , 2016, ACCV Workshops.

[24]  Bastian Leibe,et al.  Combined image- and world-space tracking in traffic scenes , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[25]  Mohan M. Trivedi,et al.  Go with the Flow: Improving Multi-view Vehicle Detection with Motion Cues , 2014, 2014 22nd International Conference on Pattern Recognition.

[26]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Rudolf Mester,et al.  Keypoint trajectory estimation using propagation based tracking , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).