Context-Based Path Prediction for Targets with Switching Dynamics

Anticipating future situations from streaming sensor data is a key perception challenge for mobile robotics and automated vehicles. We address the problem of predicting the path of objects with multiple dynamic modes. The dynamics of such targets can be described by a Switching Linear Dynamical System (SLDS). However, predictions from this probabilistic model cannot anticipate when a change in dynamic mode will occur. We propose to extract various types of cues with computer vision to provide context on the target’s behavior, and incorporate these in a Dynamic Bayesian Network (DBN). The DBN extends the SLDS by conditioning the mode transition probabilities on additional context states. We describe efficient online inference in this DBN for probabilistic path prediction, accounting for uncertainty in both measurements and target behavior. Our approach is illustrated on two scenarios in the Intelligent Vehicles domain concerning pedestrians and cyclists, so-called Vulnerable Road Users (VRUs). Here, context cues include the static environment of the VRU, its dynamic environment, and its observed actions. Experiments using stereo vision data from a moving vehicle demonstrate that the proposed approach results in more accurate path prediction than SLDS at the relevant short time horizon (1 s). It slightly outperforms a computationally more demanding state-of-the-art method.

[1]  Jean-Marc Odobez,et al.  Multiperson Visual Focus of Attention from Head Pose and Meeting Contextual Cues , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Romain Meeusen,et al.  Exposure measurement in bicycle safety analysis: A review of the literature. , 2015, Accident; analysis and prevention.

[3]  Eike Rehder,et al.  Goal-Directed Pedestrian Prediction , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[4]  Roland Siegwart,et al.  Predicting pedestrian crossing using Quantile Regression forests , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[5]  Dariu Gavrila,et al.  Analysis of pedestrian dynamics from a vehicle perspective , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[6]  Tarek Sayed,et al.  Automated Safety Diagnosis of Vehicle–Bicycle Interactions Using Computer Vision Analysis , 2013 .

[7]  Matthias Althoff,et al.  Model-Based Probabilistic Collision Detection in Autonomous Driving , 2009, IEEE Transactions on Intelligent Transportation Systems.

[8]  Dariu Gavrila,et al.  UvA-DARE ( Digital Academic Repository ) Pedestrian Path Prediction with Recursive Bayesian Filters : A Comparative Study , 2013 .

[9]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Dariu Gavrila,et al.  Active Pedestrian Safety by Automatic Braking and Evasive Steering , 2011, IEEE Transactions on Intelligent Transportation Systems.

[11]  Xavier Boyen,et al.  Tractable Inference for Complex Stochastic Processes , 1998, UAI.

[12]  Sarah Schmidt,et al.  Pedestrians at the kerb – Recognising the action intentions of humans , 2009 .

[13]  Michel Bierlaire,et al.  Behavioral Priors for Detection and Tracking of Pedestrians in Video Sequences , 2006, International Journal of Computer Vision.

[14]  Markus Braun,et al.  Pose-RCNN: Joint object detection and pose estimation using 3D object proposals , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[15]  Dariu Gavrila,et al.  Integrated pedestrian classification and orientation estimation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Hui Xiong,et al.  A new benchmark for vision-based cyclist detection , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[17]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Ian D. Reid,et al.  Guiding Visual Surveillance by Tracking Human Attention , 2009, BMVC.

[19]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Yi Yang,et al.  Intersection scan model and probability inference for vision based small-scale urban intersection detection , 2017, Intelligent Vehicles Symposium.

[21]  Feng You,et al.  Study of Bicycle Movements in Conflicts at Mixed Traffic Unsignalized Intersections , 2017, IEEE Access.

[22]  Ding Zhao,et al.  Evaluation of automated vehicles encountering pedestrians at unsignalized crossings , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[23]  Rainer Stiefelhagen,et al.  Pedestrian intention recognition using Latent-dynamic Conditional Random Fields , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[24]  S. Lauritzen Propagation of Probabilities, Means, and Variances in Mixed Graphical Association Models , 1992 .

[25]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[26]  Yoshimitsu Aoki,et al.  Danger level modeling and analysis of vehicle-pedestrian encounter using situation dependent topic model , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[27]  Philip H. S. Torr,et al.  DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Paul E. Rybski,et al.  Vision-based 3D bicycle tracking using deformable part model and Interacting Multiple Model filter , 2011, 2011 IEEE International Conference on Robotics and Automation.

[29]  Konrad Doll,et al.  Cyclists' starting behavior at intersections , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[30]  C. G. Keller,et al.  Will the Pedestrian Cross? A Study on Pedestrian Path Prediction , 2014, IEEE Transactions on Intelligent Transportation Systems.

[31]  Franz Kummert,et al.  General Behavior Prediction by a Combination of Scenario-Specific Models , 2014, IEEE Transactions on Intelligent Transportation Systems.

[32]  Markus Schreiber,et al.  LaneLoc: Lane marking based localization using highly accurate maps , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[33]  Dima Damen,et al.  Recognizing linked events: Searching the space of feasible explanations , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Dariu Gavrila,et al.  Using road topology to improve cyclist path prediction , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[35]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Michael Goldhammer,et al.  Trajectory prediction of cyclists using a physical model and an artificial neural network , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[37]  Sergiu Nedevschi,et al.  Curb Detection Based on a Multi-Frame Persistence Map for Urban Driving Scenarios , 2008, 2008 11th International IEEE Conference on Intelligent Transportation Systems.

[38]  John K. Tsotsos,et al.  Agreeing to cross: How drivers and pedestrians communicate , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[39]  Emilio Frazzoli,et al.  Intention-Aware Motion Planning , 2013, WAFR.

[40]  Hui Xiong,et al.  A Unified Framework for Concurrent Pedestrian and Cyclist Detection , 2017, IEEE Transactions on Intelligent Transportation Systems.

[41]  Martial Hebert,et al.  Activity Forecasting , 2012, ECCV.

[42]  Matthew Johnson-Roberson,et al.  Real-Time Certified Probabilistic Pedestrian Forecasting , 2017, IEEE Robotics and Automation Letters.

[43]  Dariu Gavrila,et al.  Driver and pedestrian awareness-based collision risk analysis , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[44]  Mark J. F. Gales,et al.  Rao-Blackwellised Gibbs sampling for switching linear dynamical systems , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[45]  Silvio Savarese,et al.  Knowledge Transfer for Scene-Specific Motion Prediction , 2016, ECCV.

[46]  Rainer Stiefelhagen,et al.  A Controlled Interactive Multiple Model Filter for Combined Pedestrian Intention Recognition and Path Prediction , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[47]  Pietro Perona,et al.  Pedestrian Detection: An Evaluation of the State of the Art , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Martin Lauer,et al.  3D Traffic Scene Understanding From Movable Platforms , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Kazunori Munehiro,et al.  A study on the behavior of pedestrians when confirming approach of right/left-turning vehicle while crossing a crosswalk , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[50]  Zhihan Lv,et al.  Cyclist Social Force Model at Unsignalized Intersections With Heterogeneous Traffic , 2017, IEEE Transactions on Industrial Informatics.

[51]  Li-Ta Hsu,et al.  A probabilistic model of pedestrian crossing behavior at signalized intersections for connected vehicles , 2016 .

[52]  Richard O. Duda,et al.  Use of the Hough transformation to detect lines and curves in pictures , 1972, CACM.

[53]  Dariu Gavrila,et al.  A Probabilistic Framework for Joint Pedestrian Head and Body Orientation Estimation , 2015, IEEE Transactions on Intelligent Transportation Systems.

[54]  Ulrich Brunsmann,et al.  Autonomous evasive maneuvers triggered by infrastructure-based detection of pedestrian intentions , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[55]  Dariu Gavrila,et al.  Monocular Pedestrian Detection: Survey and Experiments , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Tom Minka,et al.  Expectation Propagation for approximate Bayesian inference , 2001, UAI.

[57]  Jörg Stückler,et al.  Unsupervised Learning of Shape-Motion Patterns for Objects in Urban Street Scenes , 2016, BMVC.

[58]  Gwenn Englebienne,et al.  Mixture of Switching Linear Dynamics to Discover Behavior Patterns in Object Tracks , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[59]  James M. Rehg,et al.  Learning and Inferring Motion Patterns using Parametric Segmental Switching Linear Dynamic Systems , 2008, International Journal of Computer Vision.

[60]  Emilio Frazzoli,et al.  A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles , 2016, IEEE Transactions on Intelligent Vehicles.

[61]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[62]  Xiaogang Wang,et al.  Pedestrian Behavior Understanding and Prediction with Deep Neural Networks , 2016, ECCV.

[63]  Bolei Zhou,et al.  Scene Parsing through ADE20K Dataset , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[64]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[65]  Mohan M. Trivedi,et al.  Trajectory Learning for Activity Understanding: Unsupervised, Multilevel, and Long-Term Adaptive Approach , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[66]  Erwin de Gelder,et al.  Classification for Safety-Critical Car-Cyclist Scenarios Using Machine Learning , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[67]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[68]  Mohan M. Trivedi,et al.  Looking at Humans in the Age of Self-Driving and Highly Automated Vehicles , 2016, IEEE Transactions on Intelligent Vehicles.

[69]  Silvio Savarese,et al.  Learning Social Etiquette: Human Trajectory Understanding In Crowded Scenes , 2016, ECCV.

[70]  Jan Giebel,et al.  Shape-based pedestrian detection and tracking , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[71]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[72]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[73]  Christian Wöhler,et al.  PII: S0262-8856(98)00108-5 , 1999 .

[74]  Atsushi Yamashita,et al.  Development of pedestrian behavior model taking account of intention , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[75]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[76]  A. Kummert,et al.  The unscented Kalman filter for pedestrian tracking from a moving host , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[77]  Vladimir Pavlovic,et al.  Learning Switching Linear Models of Human Motion , 2000, NIPS.

[78]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[79]  Andreas Geiger,et al.  Object scene flow for autonomous vehicles , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[80]  Luc Van Gool,et al.  You'll never walk alone: Modeling social behavior for multi-target tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[81]  Stefano Soatto,et al.  Intent-aware long-term prediction of pedestrian motion , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[82]  Thiagalingam Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation , 2001 .

[83]  F. SergioA.Rodriguez,et al.  Context-based vector fields for multi-object tracking in application to road traffic , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[84]  Dariu Gavrila,et al.  Context-Based Pedestrian Path Prediction , 2014, ECCV.

[85]  Samuel S. Blackman,et al.  Design and Analysis of Modern Tracking Systems , 1999 .