A Field-Based Representation of Surrounding Vehicle Motion from a Monocular Camera

Sensing and presenting on-road information of moving vehicles is essential for fully and semi-automated driving. It is challenging to track vehicles from affordable on-board cameras in crowded scenes. The mismatch or missing data are unavoidable and it is ineffective to directly present uncertain cues to support the decision-making. In this paper, we propose a physical model based on incompressible fluid dynamics to represent the vehicle’s motion, which provides hints of possible collision as a continuous scalar riskmap. We estimate the position and velocity of other vehicles from a monocular on-board camera located in front of the ego-vehicle. The noisy trajectories are then modeled as the boundary conditions in the simulation of advection and diffusion process. We then interactively display the animating distribution of substances, and show that the continuous scalar riskmap well matches the perception of vehicles even in presence of the tracking failures. We test our method on real-world scenes and discuss about its application for driving assistance and autonomous vehicle in the future.

[1]  Valerio Pascucci,et al.  The Helmholtz-Hodge Decomposition—A Survey , 2013, IEEE Transactions on Visualization and Computer Graphics.

[2]  Kyongsu Yi,et al.  Design of Integrated Risk Management-Based Dynamic Driving Control of Automated Vehicles , 2017, IEEE Intelligent Transportation Systems Magazine.

[3]  Kuk-Jin Yoon,et al.  Robust Online Multi-object Tracking Based on Tracklet Confidence and Online Discriminative Appearance Learning , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Jos Stam,et al.  Stable fluids , 1999, SIGGRAPH.

[5]  Xiangyu Zhang,et al.  Light-Head R-CNN: In Defense of Two-Stage Object Detector , 2017, ArXiv.

[6]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Silvio Savarese,et al.  Learning to Track: Online Multi-object Tracking by Decision Making , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[8]  Zdenek Kalal,et al.  Tracking-Learning-Detection , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Christian Laugier,et al.  Bayesian Occupancy Filtering for Multitarget Tracking: An Automotive Application , 2006, Int. J. Robotics Res..

[10]  Joel W. Burdick,et al.  Artificial potential functions for highway driving with collision avoidance , 2008, 2008 IEEE International Conference on Robotics and Automation.

[11]  Franz Kummert,et al.  Building a probabilistic grid-based road representation from direct and indirect visual cues , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[12]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Ayoub Al-Hamadi,et al.  Stereo-Camera-Based Urban Environment Perception Using Occupancy Grid and Object Tracking , 2012, IEEE Transactions on Intelligent Transportation Systems.

[14]  Mohan M. Trivedi,et al.  Learning multi-lane trajectories using vehicle-based vision , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[15]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Robert T. Collins,et al.  Multi-target Tracking by Lagrangian Relaxation to Min-cost Network Flow , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Kuk-Jin Yoon,et al.  Robust online multiple object tracking based on the confidence-based relative motion network and correlation filter , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[18]  Mohan M. Trivedi,et al.  Dynamic Probabilistic Drivability Maps for Lane Change and Merge Driver Assistance , 2014, IEEE Transactions on Intelligent Transportation Systems.

[19]  Yi Li,et al.  Deformable Convolutional Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Dirk M. Reichardt,et al.  Collision avoidance in dynamic environments applied to autonomous vehicle guidance on the motorway , 1994, Proceedings of the Intelligent Vehicles '94 Symposium.

[21]  Konrad Schindler,et al.  Learning by Tracking: Siamese CNN for Robust Target Association , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[22]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[23]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[24]  Ming-Hsuan Yang,et al.  Online Multi-object Tracking via Structural Constraint Event Aggregation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Ramakant Nevatia,et al.  Multi-target tracking by on-line learned discriminative appearance models , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[27]  Mohan M. Trivedi,et al.  Trajectories and Maneuvers of Surrounding Vehicles With Panoramic Camera Arrays , 2016, IEEE Transactions on Intelligent Vehicles.

[28]  Robert Bridson,et al.  Fluid Simulation for Computer Graphics , 2008 .

[29]  Kaiming He,et al.  Mask R-CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[30]  PascucciValerio,et al.  The Helmholtz-Hodge Decomposition—A Survey , 2013 .

[31]  Bernt Schiele,et al.  Multi-person Tracking by Multicut and Deep Matching , 2016, ECCV Workshops.

[32]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.