Pedestrian Models for Autonomous Driving Part II: High-Level Models of Human Behavior

Abstract—Autonomous vehicles (AVs) must share space with pedestrians, both in carriageway cases such as cars at pedestrian crossings and off-carriageway cases such as delivery vehicles navigating through crowds on pedestrianized high-streets. Unlike static obstacles, pedestrians are active agents with complex, inter- active motions. Planning AV actions in the presence of pedestrians thus requires modelling of their probable future behaviour as well as detecting and tracking them. This narrative review article is Part II of a pair, together surveying the current technology stack involved in this process, organising recent research into a hierarchical taxonomy ranging from low-level image detection to high-level psychological models, from the perspective of an AV designer. This self-contained Part II covers the higher levels of this stack, consisting of models of pedestrian behaviour, from prediction of individual pedestrians’ likely destinations and paths, to game-theoretic models of interactions between pedestrians and autonomous vehicles. This survey clearly shows that, although there are good models for optimal walking behaviour, high-level psychological and social modelling of pedestrian behaviour still remains an open research question that requires many conceptual issues to be clarified. Early work has been done on descriptive and qualitative models of behaviour, but much work is still needed to translate them into quantitative algorithms for practical AV control.

[1]  Matthias Althoff,et al.  Pedestrian Models for Autonomous Driving Part I: Low-Level Models, From Sensing to Tracking , 2020, IEEE Transactions on Intelligent Transportation Systems.

[2]  Abdullah,et al.  Modeling and Simulation of Pedestrian Movement Planning Around Corners , 2019, Sustainability.

[3]  Dizan Vasquez,et al.  Novel planning-based algorithms for human motion prediction , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[4]  Satish V. Ukkusuri,et al.  Modeling of Motorist-Pedestrian Interaction at Uncontrolled Mid-block Crosswalks , 2003 .

[5]  Wolfram Burgard,et al.  Learning to predict trajectories of cooperatively navigating agents , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[6]  Mohan M. Trivedi,et al.  Learning and predicting on-road pedestrian behavior around vehicles , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[7]  Stephan Winter,et al.  Negotiation Between Vehicles and Pedestrians for the Right of Way at Intersections , 2019, IEEE Transactions on Intelligent Transportation Systems.

[8]  Henrik I. Christensen,et al.  Tracking for following and passing persons , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Tibor Petzoldt,et al.  Laboruntersuchung zur potenziellen Sicherheitswirkung einer vorderen Bremsleuchte in Pkw , 2017 .

[10]  Siddhartha S. Srinivasa,et al.  Planning-based prediction for pedestrians , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Xiaogang Wang,et al.  Pedestrian Behavior Understanding and Prediction with Deep Neural Networks , 2016, ECCV.

[12]  Xiaoou Tang,et al.  Pedestrian Attribute Recognition At Far Distance , 2014, ACM Multimedia.

[13]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[14]  Natasha Merat,et al.  Filtration analysis of pedestrian-vehicle interactions for autonomous vehicle control , 2018 .

[15]  Michael Goldhammer,et al.  Camera based pedestrian path prediction by means of polynomial least-squares approximation and multilayer perceptron neural networks , 2015, 2015 SAI Intelligent Systems Conference (IntelliSys).

[16]  Wolfram Burgard,et al.  Multimodal interaction-aware motion prediction for autonomous street crossing , 2018, Int. J. Robotics Res..

[17]  Timothy Bretl,et al.  A convex approach to inverse optimal control and its application to modeling human locomotion , 2012, 2012 IEEE International Conference on Robotics and Automation.

[18]  Friedrich M. Wahl,et al.  Acquisition of statistical motion patterns in dynamic environments and their application to mobile robot motion planning , 1997, Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robot and Systems. Innovative Robotics for Real-World Applications. IROS '97.

[19]  Vincent Lepetit,et al.  Pose estimation for category specific multiview object localization , 2009, CVPR.

[20]  Herman Herman,et al.  Comparing apples and oranges: Off‐road pedestrian detection on the National Robotics Engineering Center agricultural person‐detection dataset , 2018, J. Field Robotics.

[21]  Jianxiong Xiao,et al.  DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Zhi Yan,et al.  3DOF Pedestrian Trajectory Prediction Learned from Long-Term Autonomous Mobile Robot Deployment Data , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Keith Redmill,et al.  Top-view Trajectories: A Pedestrian Dataset of Vehicle-Crowd Interaction from Controlled Experiments and Crowded Campus , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[24]  Wenhan Luo,et al.  Multiple Object Tracking: A Review , 2014, ArXiv.

[25]  Gianni Ferretti,et al.  Generation of human walking paths , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[26]  Raúl Quintero,et al.  Pedestrian Intention and Pose Prediction through Dynamical Models and Behaviour Classification , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[27]  Serge P. Hoogendoorn,et al.  Macroscopic pedestrian flow simulation using Smoothed Particle Hydrodynamics (SPH) , 2020, Transportation Research Part C: Emerging Technologies.

[28]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Despina Stavrinos,et al.  Distraction and pedestrian safety: how talking on the phone, texting, and listening to music impact crossing the street. , 2012, Accident; analysis and prevention.

[30]  Eleonora Papadimitriou.  Theory and models of pedestrian crossing behaviour along urban trips , 2012 .

[31]  Wolfram Burgard,et al.  Learning Motion Patterns of People for Compliant Robot Motion , 2005, Int. J. Robotics Res..

[32]  Rainer Stiefelhagen,et al.  Pedestrian intention recognition using Latent-dynamic Conditional Random Fields , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[33]  Wolfram Burgard,et al.  Predicting Occupancy Distributions of Walking Humans With Convolutional Neural Networks , 2018, IEEE Robotics and Automation Letters.

[34]  Edward T. Hall,et al.  A System for the Notation of Proxemic Behavior1 , 1963 .

[35]  Bernt Schiele,et al.  CityPersons: A Diverse Dataset for Pedestrian Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Serge P. Hoogendoorn,et al.  State-of-the-art crowd motion simulation models , 2013 .

[37]  Harry Timmermans.  Pedestrian Behavior:Models, Data Collection and Applications , 2009 .

[38]  Takayuki Kanda,et al.  Modeling and Prediction of Pedestrian Behavior based on the Sub-goal Concept , 2012, Robotics: Science and Systems.

[39]  Reid G. Simmons,et al.  The curvature-velocity method for local obstacle avoidance , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[40]  Lorenza Manenti,et al.  Towards an Agent-Based Proxemic Model for Pedestrian and Group Dynamic , 2010, WOA.

[41]  D. Shinar,et al.  Pedestrians' behaviour in cross walks: the effects of fear of falling and age. , 2012, Accident; analysis and prevention.

[42]  Antonio Barrientos,et al.  Pedestrian Trajectory Prediction in Large Infrastructures - A Long-term Approach based on Path Planning , 2016, ICINCO.

[43]  Christian Vollmer,et al.  Learning to navigate through crowded environments , 2010, 2010 IEEE International Conference on Robotics and Automation.

[44]  Samantha Jamson,et al.  Behavioural Adaptation and Road Safety : Theory, Evidence and Action , 2013 .

[45]  Jörg P. Müller,et al.  Modeling Interactions of Multimodal Road Users in Shared Spaces , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[46]  Klio Monokrousou,et al.  Interpreting and Predicting Pedestrian Movement in Public Space through Space Syntax Analysis , 2016 .

[47]  Andrea Martin.  Interactive Motion Prediction using Game Theory , 2013 .

[48]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Paul Baxter,et al.  A Dataset for Action Recognition in the Wild , 2019, TAROS.

[50]  Hendrik Vermuyten,et al.  A review of optimisation models for pedestrian evacuation and design problems , 2016 .

[51]  Silvio Savarese,et al.  Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[52]  Illah R. Nourbakhsh,et al.  Planning for Human-Robot Interaction Using Time-State Aggregated POMDPs , 2008, AAAI.

[53]  Xiaoou Tang,et al.  A large-scale car dataset for fine-grained categorization and verification , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[54]  Roger L. Hughes,et al.  A continuum theory for the flow of pedestrians , 2002 .

[55]  Martin Buss,et al.  Understanding Human Avoidance Behavior: Interaction-Aware Decision Making Based on Game Theory , 2016, Int. J. Soc. Robotics.

[56]  Stefania Bandini,et al.  A simulation model for non-signalized pedestrian crosswalks based on evidence from on field observation , 2017, Intelligenza Artificiale.

[57]  Michael Batty,et al.  Predicting where we walk , 1997, Nature.

[58]  Kris M. Kitani,et al.  Forecasting Interactive Dynamics of Pedestrians with Fictitious Play , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Dietmar Bauer,et al.  Macroscopic pedestrian flow simulation for designing crowd control measures in public transport after special events , 2007, SCSC.

[60]  Klaus Bengler,et al.  A Pedestrian Simulator for Urban Crossing Scenarios , 2016 .

[61]  Stefan Roth,et al.  MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking , 2015, ArXiv.

[62]  Reid G. Simmons,et al.  Natural person-following behavior for social robots , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[63]  Xiaogang Wang,et al.  Locally Aligned Feature Transforms across Views , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[64]  Dietmar Bauer,et al.  Comparing pedestrian movement simulation models for a crossing area based on real world data , 2011 .

[65]  Christopher K. I. Williams,et al.  Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .

[66]  Lily Elefteriadou,et al.  Empirically-Based Performance Assessment and Simulation of Pedestrian Behavior at Unsignalized Crossings , 2014 .

[67]  Matthias Althoff,et al.  Set-Based Prediction of Pedestrians in Urban Environments Considering Formalized Traffic Rules , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[68]  Roland Siegwart,et al.  Inferring Pedestrian Motions at Urban Crosswalks , 2019, IEEE Transactions on Intelligent Transportation Systems.

[69]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[70]  Ruigang Yang,et al.  The ApolloScape Open Dataset for Autonomous Driving and Its Application , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[71]  Rikard Fredriksson,et al.  Will There Be New Communication Needs When Introducing Automated Vehicles to the Urban Context , 2017 .

[72]  Lorenzo Domenichini,et al.  Drivers' performance in response to engineering treatments at pedestrian crossings , 2018 .

[73]  E. Hall,et al.  The Hidden Dimension , 1970 .

[74]  Kazuhiko Kawamoto,et al.  Learning pedestrian dynamics with kriging , 2016, 2016 IEEE/ACIS 15th International Conference on Computer and Information Science (ICIS).

[75]  Otmar Hilliges,et al.  Learning Human Motion Models for Long-Term Predictions , 2017, 2017 International Conference on 3D Vision (3DV).

[76]  Miho Iryo-Asano,et al.  Analysis of Pedestrian Clearance Time at Signalized Crosswalks in Japan , 2014, ANT/SEIT.

[77]  Sowmya Somanath,et al.  Can Interfaces Facilitate Communication in Autonomous Vehicle-Pedestrian Interaction? , 2018, HRI.

[78]  Mubarak Shah,et al.  A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[79]  Andreas Krause,et al.  Unfreezing the robot: Navigation in dense, interacting crowds , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[80]  John K. Tsotsos,et al.  Autonomous Vehicles That Interact With Pedestrians: A Survey of Theory and Practice , 2018, IEEE Transactions on Intelligent Transportation Systems.

[81]  P. Norman,et al.  Predicting adolescent pedestrians' road-crossing intentions: an application and extension of the Theory of Planned Behaviour. , 2003, Health education research.

[82]  Natasha Merat,et al.  Designing the interaction of automated vehicles with other traffic participants: A design framework based on human needs and expectations , 2018 .

[83]  Angelos Amditis,et al.  Field Observations of Interactions Among Drivers at Unsignalized Urban Intersections , 2018 .

[84]  Anne-Hélène Olivier,et al.  Walking with Virtual People: Evaluation of Locomotion Interfaces in Dynamic Environments , 2018, IEEE Transactions on Visualization and Computer Graphics.

[85]  Yang Gao,et al.  End-to-End Learning of Driving Models from Large-Scale Video Datasets , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[86]  Natasha Merat,et al.  When Should the Chicken Cross the Road? - Game Theory for Autonomous Vehicle - Human Interactions , 2018, VEHITS.

[87]  Leah L Thompson,et al.  Impact of social and technological distraction on pedestrian crossing behaviour: an observational study , 2012, Injury Prevention.

[88]  Rikard Fredriksson,et al.  Communicating Intent of Automated Vehicles to Pedestrians , 2018, Front. Psychol..

[89]  Geoffrey E. Hinton,et al.  Modeling Human Motion Using Binary Latent Variables , 2006, NIPS.

[90]  Xiaogang Wang,et al.  DeepReID: Deep Filter Pairing Neural Network for Person Re-identification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[91]  John K. Tsotsos,et al.  Agreeing to cross: How drivers and pedestrians communicate , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[92]  Peter Christiansen,et al.  FieldSAFE: Dataset for Obstacle Detection in Agriculture , 2017, Sensors.

[93]  Wael K.M. Alhajyaseen,et al.  Studying critical pedestrian behavioral changes for the safety assessment at signalized crosswalks , 2017 .

[94]  Ulrich Kressel,et al.  Probabilistic trajectory prediction with Gaussian mixture models , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[95]  Christian Laugier,et al.  From Proxemics Theory to Socially-Aware Navigation: A Survey , 2014, International Journal of Social Robotics.

[96]  J. Slob,et al.  State-of-the-Art Driving Simulators, a Literature Survey , 2022 .

[97]  Adam Astrid Kemperman,et al.  Modeling pedestrian movement in shopping street segments , 2009 .

[98]  Nicola Bellomo,et al.  On the Modeling of Traffic and Crowds: A Survey of Models, Speculations, and Perspectives , 2011, SIAM Rev..

[99]  Malte Risto,et al.  Human-Vehicle Interfaces: The Power of Vehicle Movement Gestures in Human Road User Coordination , 2017 .

[100]  Henrik I. Christensen,et al.  Autonomous person following for telepresence robots , 2013, 2013 IEEE International Conference on Robotics and Automation.

[101]  Xiaogang Wang,et al.  Human Reidentification with Transferred Metric Learning , 2012, ACCV.

[102]  Marc Hanheide,et al.  A Computational Model of Human-Robot Spatial Interactions Based on a Qualitative Trajectory Calculus , 2015, Robotics.

[103]  Francesco Solera,et al.  Performance Measures and a Data Set for Multi-target, Multi-camera Tracking , 2016, ECCV Workshops.

[104]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[105]  W. A. Harrell,et al.  The Impact of Pedestrian Visibility and Assertiveness on Motorist Yielding , 1993 .

[106]  Jonathan P. How,et al.  Context-Aware Pedestrian Motion Prediction In Urban Intersections , 2018, ArXiv.

[107]  Xiaogang Jin,et al.  Force-based Heterogeneous Traffic Simulation for Autonomous Vehicle Testing , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[108]  William H. Warren,et al.  A behavioral dynamics approach to modeling realistic pedestrian behavior , 2012 .

[109]  Dizan Vasquez,et al.  A survey on motion prediction and risk assessment for intelligent vehicles , 2014, ROBOMECH Journal.

[110]  Michel Bierlaire,et al.  Discrete Choice Models for Pedestrian Walking Behavior , 2006 .

[111]  Hideki Nakamura,et al.  Analysis and Modeling of Pedestrian Crossing Behavior During the Pedestrian Flashing Green Interval , 2015, IEEE Transactions on Intelligent Transportation Systems.

[112]  Bing-Hong Wang,et al.  Modeling pedestrian evacuation by means of game theory , 2017 .

[113]  Miho Iryo-Asano,et al.  Modeling pedestrian crossing speed profiles considering speed change behavior for the safety assessment of signalized intersections. , 2017, Accident; analysis and prevention.

[114]  Hesham Rakha,et al.  Agent-Based Game Theory Modeling for Driverless Vehicles at Intersections , 2013 .

[115]  Toby Sharp,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR.

[116]  George Yannis,et al.  A critical assessment of pedestrian behaviour models , 2009 .

[117]  Lee Lacy,et al.  Defense Advanced Research Projects Agency (DARPA) Agent Markup Language Computer Aided Knowledge Acquisition , 2005 .

[118]  Tom Duckett,et al.  Online learning for 3D LiDAR-based human detection: experimental analysis of point cloud clustering and classification methods , 2019, Autonomous Robots.

[119]  Anthony G. Cohn,et al.  Unsupervised Learning of Qualitative Motion Behaviours by a Mobile Robot , 2016, AAMAS.

[120]  John K. Tsotsos,et al.  Joint Attention in Driver-Pedestrian Interaction: from Theory to Practice , 2018, ArXiv.

[121]  Anthony G. Cohn,et al.  Latent Dirichlet Allocation for Unsupervised Activity Analysis on an Autonomous Mobile Robot , 2017, AAAI.

[122]  Wolfram Burgard,et al.  Learning motion patterns of persons for mobile service robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[123]  Matteo Munaro,et al.  Fast RGB-D people tracking for service robots , 2014, Auton. Robots.

[124]  Rune Elvik,et al.  A review of game-theoretic models of road user behaviour. , 2014, Accident; analysis and prevention.

[125]  H. Timmermans,et al.  A Model of Pedestrian Route Choice and Demand for Retail Facilities within Inner-City Shopping Areas , 2010 .

[126]  Mohan M. Trivedi,et al.  Trajectory Forecasts in Unknown Environments Conditioned on Grid-Based Plans , 2020, ArXiv.

[127]  Xiaogang Wang,et al.  Understanding collective crowd behaviors: Learning a Mixture model of Dynamic pedestrian-Agents , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[128]  Serge P. Hoogendoorn,et al.  Pedestrian route-choice and activity scheduling theory and models , 2004 .

[129]  Axel Klar,et al.  Particle methods for pedestrian flow models: From microscopic to nonlocal continuum models , 2014 .

[130]  Sowmya Somanath,et al.  Communicating Awareness and Intent in Autonomous Vehicle-Pedestrian Interaction , 2018, CHI.

[131]  Matthias Althoff,et al.  Reachability Analysis and its Application to the Safety Assessment of Autonomous Cars , 2010 .

[132]  Dinesh Manocha,et al.  GLMP- realtime pedestrian path prediction using global and local movement patterns , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[133]  Yi He,et al.  Pedestrian-vehicular collision avoidance based on vision system , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[134]  Jonathan Krause,et al.  3D Object Representations for Fine-Grained Categorization , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[135]  Takeo Igarashi,et al.  Eyes on a Car: an Interface Design for Communication between an Autonomous Car and a Pedestrian , 2017, AutomotiveUI.

[136]  I. M. Bernhoft,et al.  Preferences and behaviour of pedestrians and cyclists by age and gender , 2008 .

[137]  Martin Lauer,et al.  A Literature Review on the Prediction of Pedestrian Behavior in Urban Scenarios , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[138]  Marc Hanheide,et al.  Real-time multisensor people tracking for human-robot spatial interaction , 2015 .

[139]  Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles , 2022 .

[140]  Jean-Paul Laumond,et al.  An Optimality Principle Governing Human Walking , 2008, IEEE Transactions on Robotics.

[141]  Stefan Roth,et al.  MOT16: A Benchmark for Multi-Object Tracking , 2016, ArXiv.

[142]  Seul Jung,et al.  Pedestrian trajectory prediction via the Social‐Grid LSTM model , 2018, The Journal of Engineering.

[143]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[144]  David Hsu,et al.  Intention-aware online POMDP planning for autonomous driving in a crowd , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[145]  Gang Ren,et al.  A cellular automaton simulation model for pedestrian and vehicle interaction behaviors at unsignalized mid-block crosswalks. , 2016, Accident; analysis and prevention.

[146]  Qiang Xu,et al.  nuScenes: A Multimodal Dataset for Autonomous Driving , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[147]  Annette Werner,et al.  New Colours for Autonomous Driving: An Evaluation of Chromaticities for the External Lighting Equipment of Autonomous Vehicles , 2018 .

[148]  Jean-Paul Laumond,et al.  On the nonholonomic nature of human locomotion , 2008, Auton. Robots.

[149]  Franz Kummert,et al.  Pedestrian crossing prediction using multiple context-based models , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[150]  Masayoshi Tomizuka,et al.  INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps , 2019, ArXiv.

[151]  Simon Lucey,et al.  Argoverse: 3D Tracking and Forecasting With Rich Maps , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[152]  Luis Salgado,et al.  Video analysis-based vehicle detection and tracking using an MCMC sampling framework , 2012, EURASIP J. Adv. Signal Process..

[153]  Miguel Ángel Sotelo,et al.  Pedestrian Path, Pose, and Intention Prediction Through Gaussian Process Dynamical Models and Pedestrian Activity Recognition , 2019, IEEE Transactions on Intelligent Transportation Systems.

[154]  Jitendra Malik,et al.  Recurrent Network Models for Human Dynamics , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[155]  Serge P. Hoogendoorn,et al.  Simulation of pedestrian flows by optimal control and differential games , 2003 .

[156]  A. Hamish Jamson,et al.  Whatever Happened to the LADS? Design and Development of the New University of Leeds Driving Simulator , 2007 .

[157]  Carol Holland,et al.  The effect of age, gender and driver status on pedestrians' intentions to cross the road in risky situations. , 2007, Accident; analysis and prevention.

[158]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[159]  Marcello Pelillo,et al.  A Game-Theoretic Probabilistic Approach for Detecting Conversational Groups , 2014, ACCV.

[160]  Gunnar G. Løvås,et al.  Modeling and Simulation of Pedestrian Traffic Flow , 1994 .

[161]  P G Gipps,et al.  A micro simulation model for pedestrian flows , 1985 .

[162]  Silvio Savarese,et al.  Learning Social Etiquette: Human Trajectory Understanding In Crowded Scenes , 2016, ECCV.

[163]  Julien Pettré,et al.  Social Ways: Learning Multi-Modal Distributions of Pedestrian Trajectories With GANs , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[164]  Atsushi Yamashita,et al.  Development of pedestrian behavior model taking account of intention , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[165]  Luc Van Gool,et al.  Predicting Pedestrian Trajectories , 2011, Visual Analysis of Humans.

[166]  Kardi Teknomo,et al.  Microscopic Pedestrian Flow Characteristics: Development of an Image Processing Data Collection and Simulation Model , 2016, ArXiv.

[167]  Vladlen Koltun,et al.  Playing for Benchmarks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[168]  Alexandre Alahi,et al.  Adversarial Loss for Human Trajectory Prediction , 2022 .

[169]  Natasha Merat,et al.  Models of Human Decision-Making as Tools for Estimating and Optimizing Impacts of Vehicle Automation , 2018, Transportation Research Record: Journal of the Transportation Research Board.

[170]  Dariu Gavrila,et al.  UvA-DARE ( Digital Academic Repository ) Pedestrian Path Prediction with Recursive Bayesian Filters : A Comparative Study , 2013 .

[171]  Mohan M. Trivedi,et al.  Trajectory analysis and prediction for improved pedestrian safety: Integrated framework and evaluations , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[172]  Nicolas Guéguen,et al.  A pedestrian’s stare and drivers’ stopping behavior: A field experiment at the pedestrian crossing , 2015 .

[173]  Stefania Bandini,et al.  Observation results on pedestrian-vehicle interactions at non-signalized intersections towards simulation , 2018, Transportation Research Part F: Traffic Psychology and Behaviour.

[174]  Simon Parsons,et al.  Principles of Robot Motion: Theory, Algorithms and Implementations by Howie Choset, Kevin M. Lynch, Seth Hutchinson, George Kantor, Wolfram Burgard, Lydia E. Kavraki and Sebastian Thrun, 603 pp., $60.00, ISBN 0-262-033275 , 2007, The Knowledge Engineering Review.

[175]  Kate Saenko,et al.  Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoning , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[176]  Dani Lischinski,et al.  Crowds by Example , 2007, Comput. Graph. Forum.

[177]  Ko Yamamoto,et al.  Continuum model of crossing pedestrian flows and swarm control based on temporal/spatial frequency , 2011, 2011 IEEE International Conference on Robotics and Automation.

[178]  Carol Holland,et al.  Gender differences in factors predicting unsafe crossing decisions in adult pedestrians across the lifespan: a simulation study. , 2010, Accident; analysis and prevention.

[179]  M. Shah,et al.  Abnormal crowd behavior detection using social force model , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[180]  Natasha Merat,et al.  Towards pedestrian-AV interaction: method for elucidating pedestrian preferences , 2018 .

[181]  Wendy Ju,et al.  Expressing thought: Improving robot readability with animation principles , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[182]  Alan Kingstone,et al.  The effects of personal music devices on pedestrian behaviour , 2012 .

[183]  Kai Oliver Arras,et al.  Joint Long-Term Prediction of Human Motion Using a Planning-Based Social Force Approach , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[184]  A. Katz,et al.  An Experimental Study of Driver and Pedestrian Interaction during the Crossing Conflict , 1975 .

[185]  Cristian Giacomini,et al.  Survey of detection techniques, mathematical models and simulation software in pedestrian dynamics , 2017 .

[186]  Victor J. Blue,et al.  Cellular automata microsimulation for modeling bi-directional pedestrian walkways , 2001 .

[187]  B. Fildes,et al.  An investigation of road crossing behaviour of older pedestrians , 1995 .

[188]  Alexandre Alahi,et al.  Human Trajectory Prediction using Adversarial Loss , 2019 .

[189]  D. Manocha,et al.  PedLearn : Realtime Pedestrian Tracking , Behavior Learning , and Navigation for Autonomous Vehicles , 2017 .

[190]  Dan Roth,et al.  Learning to detect objects in images via a sparse, part-based representation , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[191]  Ian D. Reid,et al.  Stable multi-target tracking in real-time surveillance video , 2011, CVPR 2011.

[192]  Luc Van Gool,et al.  Depth and Appearance for Mobile Scene Analysis , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[193]  Anind K. Dey,et al.  Maximum Entropy Inverse Reinforcement Learning , 2008, AAAI.

[194]  Kathrin Maria Gerling,et al.  Virtual reality crowd simulation: effects of agent density on user experience and behaviour , 2018, Virtual Reality.

[195]  Paul Newman,et al.  1 year, 1000 km: The Oxford RobotCar dataset , 2017, Int. J. Robotics Res..

[196]  M Suzanne Zeedyk,et al.  Behavioural observations of adult-child pairs at pedestrian crossings. , 2003, Accident; analysis and prevention.

[197]  Stefano Soatto,et al.  Intent-aware long-term prediction of pedestrian motion , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[198]  Barbara Majecka,et al.  Statistical models of pedestrian behaviour in the Forum , 2009 .

[199]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[200]  O. Watson,et al.  Quantitative Research in Proxemic Behavior1 , 1966 .

[201]  Viola Cavallo,et al.  The effects of aging on street-crossing behavior: from estimation to actual crossing. , 2009, Accident; analysis and prevention.

[202]  Sowmya Somanath,et al.  AV-Pedestrian Interaction Design Using a Pedestrian Mixed Traffic Simulator , 2019, Conference on Designing Interactive Systems.

[203]  Aurélie Dommes,et al.  Review of safety and mobility issues among older pedestrians. , 2016, Accident; analysis and prevention.

[204]  Eike Rehder,et al.  Goal-Directed Pedestrian Prediction , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[205]  Rainer Stiefelhagen,et al.  A Controlled Interactive Multiple Model Filter for Combined Pedestrian Intention Recognition and Path Prediction , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[206]  Harry J. P. Timmermans,et al.  A Multi-Agent Cellular Automata System for Visualising Simulated Pedestrian Activity , 2000, ACRI.

[207]  Martial Hebert,et al.  Activity Forecasting , 2012, ECCV.

[208]  Dan Roth,et al.  Learning a Sparse Representation for Object Detection , 2002, ECCV.

[209]  Natasha Merat,et al.  Predicting pedestrian road-crossing assertiveness for autonomous vehicle control , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[210]  Ryosuke Shibasaki,et al.  Pedestrian Trajectory Prediction in Extremely Crowded Scenarios , 2019, Sensors.

[211]  Julian Bock,et al.  Self-learning Trajectory Prediction with Recurrent Neural Networks at Intelligent Intersections , 2017, VEHITS.

[212]  Davide Castelvecchi,et al.  Can we open the black box of AI? , 2016, Nature.

[213]  Brett R. Fajen,et al.  Behavioral dynamics of steering, obstacle avoidance, and route selection. , 2003 .

[214]  Jonas Andersson,et al.  Hello Human, can you read my mind? , 2017, ERCIM News.

[215]  Michael P. Clamann,et al.  Evaluation of Vehicle-to-Pedestrian Communication Displays for Autonomous Vehicles , 2017 .

[216]  Wei Liu,et al.  Long-Term Human Motion Prediction by Modeling Motion Context and Enhancing Motion Dynamic , 2018, IJCAI.

[217]  Wolfgang Hübner,et al.  Particle-based Pedestrian Path Prediction using LSTM-MDL Models , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[218]  Matthias Althoff,et al.  Probabilistic Map-based Pedestrian Motion Prediction Taking Traffic Participants into Consideration , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[219]  Stefania Bandini,et al.  Studying Pedestrian and Crowd Dynamics through Integrated Analysis and Synthesis , 2013, IEEE Intelligent Systems.

[220]  Martin Buss,et al.  Interactive navigation of humans from a game theoretic perspective , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[221]  Roland Siegwart,et al.  Feature Relevance Estimation for Learning Pedestrian Behavior at Crosswalks , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[222]  Dirk Helbing,et al.  How simple rules determine pedestrian behavior and crowd disasters , 2011, Proceedings of the National Academy of Sciences.

[223]  Dariu Gavrila,et al.  Context-Based Pedestrian Path Prediction , 2014, ECCV.

[224]  Angelos Amditis,et al.  Naturalistic Observation of Interactions Between Car Drivers and Pedestrians in High Density Urban Settings , 2018, Advances in Intelligent Systems and Computing.

[225]  Guizhen Yu,et al.  Assessing right-turning vehicle-pedestrian conflicts at intersections using an integrated microscopic simulation model. , 2019, Accident; analysis and prevention.

[226]  Michael Batty,et al.  Pedestrian Behaviour Modelling , 2004 .

[227]  Christina L. Bloebaum,et al.  Modeling Decision and Game Theory Based Pedestrian Velocity Vector Decisions with Interacting Individuals , 2016 .

[228]  Bernt Schiele,et al.  Long-Term On-Board Prediction of Pedestrians in Traffic Scenes , 2017 .

[229]  Alexandra Kirsch,et al.  Towards Legible Robot Navigation - How to Increase the Intend Expressiveness of Robot Navigation Behavior , 2013, ICSR 2013.

[230]  Girish Chowdhary,et al.  Intent Communication between Autonomous Vehicles and Pedestrians , 2017, ArXiv.

[231]  B. Schiele,et al.  Multi-cue onboard pedestrian detection , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[232]  Wendy Ju,et al.  Ghost driver: A field study investigating the interaction between pedestrians and driverless vehicles , 2016, 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[233]  G B Grayson,et al.  AGE-RELATED DIFFERENCES IN THE ROAD CROSSING BEHAVIOUR OF ADULT PEDESTRIANS , 1980 .

[234]  Mohan M. Trivedi,et al.  A Survey of Vision-Based Trajectory Learning and Analysis for Surveillance , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[235]  Luc Van Gool,et al.  You'll never walk alone: Modeling social behavior for multi-target tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[236]  Qi Tian,et al.  Scalable Person Re-identification: A Benchmark , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[237]  Michael J. Black,et al.  On Human Motion Prediction Using Recurrent Neural Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[238]  Martin Lauer,et al.  Pedestrian Prediction by Planning Using Deep Neural Networks , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[239]  Leon Sterling,et al.  A conceptual framework for specifying and developing pedestrian models , 2005 .

[240]  Alan Penn,et al.  Encoding Natural Movement as an Agent-Based System: An Investigation into Human Pedestrian Behaviour in the Built Environment , 2002 .

[241]  Philip H. S. Torr,et al.  DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[242]  Natasha Merat,et al.  Towards game theoretic AV controllers: measuring pedestrian behaviour in Virtual Reality , 2019 .

[243]  Anne Spalanzani,et al.  Natural vision based method for predicting pedestrian behaviour in urban environments , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[244]  Dariu Gavrila,et al.  Monocular Pedestrian Detection: Survey and Experiments , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[245]  Natasha Merat,et al.  Empirical game theory of pedestrian interaction for autonomous vehicles , 2018 .

[246]  Natasha Merat,et al.  A heuristic model for pedestrian intention estimation , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[247]  Silvio Savarese,et al.  SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[248]  Yi Yang,et al.  Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in Vitro , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[249]  Eric P. Xing,et al.  Real-to-Virtual Domain Unification for End-to-End Autonomous Driving , 2018, ECCV.

[250]  Hubert Klüpfel,et al.  Evacuation Dynamics: Empirical Results, Modeling and Applications , 2009, Encyclopedia of Complexity and Systems Science.

[251]  Juergen Gall,et al.  PoseTrack: Joint Multi-person Pose Estimation and Tracking , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[252]  Alex Fridman,et al.  Hacking Nonverbal Communication Between Pedestrians and Vehicles in Virtual Reality , 2019, Proceedings of the 10th International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design: driving assessment 2019.