A Literature Review on the Prediction of Pedestrian Behavior in Urban Scenarios

The ability to anticipate pedestrian actions on streets is a safety issue for intelligent cars and has increasingly drawn the attention of the automotive industry. Estimating when pedestrians will cross streets has proved a challenging task, since they can move in many different directions, suddenly change motion, be occluded by a variety of obstacles and distracted while talking to other pedestrians or typing on a mobile phone. Moreover, their decisions can also be affected by several factors. This paper explores the ways pedestrians' intention estimation has been studied, evaluated, and evolved. It provides a literature review on pedestrian behavior prediction, addresses available solutions, state-of-the-art developments, and hurdles to be overcome towards reaching a solution that is closer to the human ability to predict and interpret such scenarios. Although many studies can precisely estimate pedestrians' positioning one second before they cross a street, most of them cannot precisely predict when they will stop at a curb.

[1]  Bilal Farooq,et al.  Distracted pedestrians crossing behaviour: Application of immersive head mounted virtual reality , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[2]  Klaus C. J. Dietmayer,et al.  Stationary Detection of the Pedestrian?s Intention at Intersections , 2013, IEEE Intelligent Transportation Systems Magazine.

[3]  Sarah Schmidt,et al.  Pedestrians at the kerb – Recognising the action intentions of humans , 2009 .

[4]  Qixiang Ye,et al.  Pedestrian Detection in Video Images via Error Correcting Output Code Classification of Manifold Subclasses , 2012, IEEE Transactions on Intelligent Transportation Systems.

[5]  J. Ferryman,et al.  PETS2009: Dataset and challenge , 2009, 2009 Twelfth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance.

[6]  Julian Bock,et al.  Self-learning Trajectory Prediction with Recurrent Neural Networks at Intelligent Intersections , 2017, VEHITS.

[7]  Pascal Fua,et al.  Multicamera People Tracking with a Probabilistic Occupancy Map , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Martin Lauer,et al.  Pedestrian Prediction by Planning Using Deep Neural Networks , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[9]  Robert Fitch,et al.  Bayesian intention inference for trajectory prediction with an unknown goal destination , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[10]  Swarup Medasani,et al.  Active safety and collision alerts using Contextual Visual Dataspace , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[11]  Eike Rehder,et al.  Goal-Directed Pedestrian Prediction , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[12]  Roland Siegwart,et al.  Predicting pedestrian crossing using Quantile Regression forests , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[13]  Dariu Gavrila,et al.  Analysis of pedestrian dynamics from a vehicle perspective , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[14]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[15]  Zsolt Kira,et al.  Fusing LIDAR and images for pedestrian detection using convolutional neural networks , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[16]  Kang-Hyun Jo,et al.  Detection of pedestrian crossing road using action classification model , 2015, 2015 IEEE International Conference on Advanced Intelligent Mechatronics (AIM).

[17]  Rainer Stiefelhagen,et al.  Pedestrian intention recognition using Latent-dynamic Conditional Random Fields , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[18]  J. Mazziotta,et al.  Grasping the Intentions of Others with One's Own Mirror Neuron System , 2005, PLoS biology.

[19]  Atsushi Yamashita,et al.  Development of pedestrian behavior model taking account of intention , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Sheng-Fuu Lin,et al.  Pedestrians and vehicles recognition based on image recognition and laser distance detection , 2016, 2016 16th International Conference on Control, Automation and Systems (ICCAS).

[21]  Dariu Gavrila,et al.  Driver and pedestrian awareness-based collision risk analysis , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[22]  Li-Ta Hsu,et al.  A Probabilistic Model for the Estimation of Pedestrian Crossing Behavior at Signalized Intersections , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[23]  Luca Mantecchini,et al.  Empirical Analysis of Pedestrian Delay Models at Urban Intersections , 2015 .

[24]  Roland Siegwart,et al.  Feature Relevance Estimation for Learning Pedestrian Behavior at Crosswalks , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[25]  Jonathan P. How,et al.  Real-Time Predictive Modeling and Robust Avoidance of Pedestrians with Uncertain, Changing Intentions , 2014, WAFR.

[26]  C. G. Keller,et al.  Will the Pedestrian Cross? A Study on Pedestrian Path Prediction , 2014, IEEE Transactions on Intelligent Transportation Systems.

[27]  Mohammed S. Tarawneh,et al.  Evaluation of pedestrian speed in Jordan with investigation of some contributing factors , 2001 .

[28]  Li-Ta Hsu,et al.  Probability estimation for pedestrian crossing intention at signalized crosswalks , 2015, 2015 IEEE International Conference on Vehicular Electronics and Safety (ICVES).

[29]  B. Schiele,et al.  How Far are We from Solving Pedestrian Detection? , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Bernt Schiele,et al.  Monocular 3D pose estimation and tracking by detection , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Roland Siegwart,et al.  A data-driven approach for pedestrian intention estimation , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[32]  Raúl Quintero,et al.  Pedestrian path prediction based on body language and action classification , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[33]  Hideki Nakamura,et al.  Application of social force model to pedestrian behavior analysis at signalized crosswalk , 2014 .

[34]  Brendan Tran Morris,et al.  Observing behaviors at intersections: A review of recent studies & developments , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[35]  Luc Van Gool,et al.  You'll never walk alone: Modeling social behavior for multi-target tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[36]  Stefano Soatto,et al.  Intent-aware long-term prediction of pedestrian motion , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[37]  John K. Tsotsos,et al.  Are They Going to Cross? A Benchmark Dataset and Baseline for Pedestrian Crosswalk Behavior , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[38]  Kang-Hyun Jo,et al.  Detection of pedestrian crossing road: A study on pedestrian pose recognition , 2017, Neurocomputing.

[39]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Lisheng Jin,et al.  Study on vehicle front pedestrian detection based on 3D laser scanner , 2011, Proceedings 2011 International Conference on Transportation, Mechanical, and Electrical Engineering (TMEE).

[41]  Tarak Gandhi,et al.  Computer Vision and Machine Learning for Enhancing Pedestrian Safety , 2008, Computational Intelligence in Automotive Applications.

[42]  Dariu Gavrila,et al.  Multi-cue Pedestrian Detection and Tracking from a Moving Vehicle , 2007, International Journal of Computer Vision.

[43]  Keiichi Yamada,et al.  Estimation of street crossing intention from a pedestrian's posture on a sidewalk using multiple image frames , 2011, The First Asian Conference on Pattern Recognition.

[44]  Miguel Angel Sotelo,et al.  Assistive Intelligent Transportation Systems: The Need for User Localization and Anonymous Disability Identification , 2017, IEEE Intelligent Transportation Systems Magazine.

[45]  Pascal Fua,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Multiple Object Tracking Using K-shortest Paths Optimization , 2022 .

[46]  Tamitza Toroyan,et al.  Global status report on road safety , 2009, Injury Prevention.

[47]  Rainer Stiefelhagen,et al.  A Controlled Interactive Multiple Model Filter for Combined Pedestrian Intention Recognition and Path Prediction , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[48]  U. Lachapelle,et al.  "Outta my way!" Individual and environmental correlates of interactions between pedestrians and vehicles during street crossings. , 2017, Accident; analysis and prevention.

[49]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Li-Ta Hsu,et al.  Motion planning based on learning models of pedestrian and driver behaviors , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[51]  Qiang Li,et al.  Indoor Pedestrian Trajectory Detection with LSTM Network , 2017, 22017 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC).

[52]  Dariu Gavrila,et al.  UvA-DARE ( Digital Academic Repository ) Pedestrian Path Prediction with Recursive Bayesian Filters : A Comparative Study , 2013 .

[53]  Massimo Bertozzi,et al.  360° Detection and tracking algorithm of both pedestrian and vehicle using fisheye images , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[54]  Kang-Hyun Jo,et al.  Estimation of walking direction for pedestrian path prediction from moving vehicle , 2015, 2015 IEEE/SICE International Symposium on System Integration (SII).

[55]  Christoph Stiller,et al.  Comparison and evaluation of pedestrian motion models for vehicle safety systems , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[56]  Dariu Gavrila,et al.  Active Pedestrian Safety by Automatic Braking and Evasive Steering , 2011, IEEE Transactions on Intelligent Transportation Systems.

[57]  Klaus C. J. Dietmayer,et al.  Real-time detection and tracking of pedestrians at intersections using a network of laserscanners , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[58]  Michael Goldhammer,et al.  Camera based pedestrian path prediction by means of polynomial least-squares approximation and multilayer perceptron neural networks , 2015, 2015 SAI Intelligent Systems Conference (IntelliSys).

[59]  Hui Xiong,et al.  A Unified Framework for Concurrent Pedestrian and Cyclist Detection , 2017, IEEE Transactions on Intelligent Transportation Systems.

[60]  Martial Hebert,et al.  Activity Forecasting , 2012, ECCV.

[61]  Raúl Quintero,et al.  Pedestrian Intention and Pose Prediction through Dynamical Models and Behaviour Classification , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[62]  Berthold Färber,et al.  Communication and Communication Problems Between Autonomous Vehicles and Human Drivers , 2016 .

[63]  Mohan M. Trivedi,et al.  Trajectory analysis and prediction for improved pedestrian safety: Integrated framework and evaluations , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[64]  Dariu Gavrila,et al.  Monocular Pedestrian Detection: Survey and Experiments , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[65]  Kang-Hyun Jo,et al.  Pedestrian action recognition using motion type classification , 2015, 2015 IEEE 2nd International Conference on Cybernetics (CYBCONF).

[66]  Dariu Gavrila,et al.  Will the Pedestrian Cross? Probabilistic Path Prediction Based on Learned Motion Features , 2011, DAGM-Symposium.

[67]  Franz Kummert,et al.  Pedestrian crossing prediction using multiple context-based models , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[68]  Bernt Schiele,et al.  Ten Years of Pedestrian Detection, What Have We Learned? , 2014, ECCV Workshops.

[69]  Mohan M. Trivedi,et al.  Looking at Humans in the Age of Self-Driving and Highly Automated Vehicles , 2016, IEEE Transactions on Intelligent Vehicles.

[70]  Mohan M. Trivedi,et al.  Learning and predicting on-road pedestrian behavior around vehicles , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[71]  Klaus C. J. Dietmayer,et al.  Stereo-Vision-Based Pedestrian's Intention Detection in a Moving Vehicle , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[72]  ByoungChul Ko,et al.  Pedestrian intention prediction based on dynamic fuzzy automata for vehicle driving at nighttime , 2017 .

[73]  Julian F. P. Kooij,et al.  Supplemental Material Context-based Pedestrian Path Prediction , 2014 .

[74]  Raúl Quintero,et al.  Pedestrian intention recognition by means of a Hidden Markov Model and body language , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[75]  A. Broggi,et al.  Pedestrian localization and tracking system with Kalman filtering , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[76]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[77]  Klaus C. J. Dietmayer,et al.  Early detection of the Pedestrian's intention to cross the street , 2012, 2012 15th International IEEE Conference on Intelligent Transportation Systems.

[78]  Anne Spalanzani,et al.  Natural vision based method for predicting pedestrian behaviour in urban environments , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[79]  Luc Van Gool,et al.  A mobile vision system for robust multi-person tracking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[80]  Michael Goldhammer,et al.  Early prediction of a pedestrian's trajectory at intersections , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[81]  Yutaka Satoh,et al.  Fine-Grained Walking Activity Recognition via Driving Recorder Dataset , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[82]  Miguel Cazorla,et al.  Pedestrian Movement Direction Recognition Using Convolutional Neural Networks , 2017, IEEE Transactions on Intelligent Transportation Systems.