Explaining human interactions on the road by large-scale integration of computational psychological theory

Abstract When humans share space in road traffic, as drivers or as vulnerable road users, they draw on their full range of communicative and interactive capabilities. Much remains unknown about these behaviors, but they need to be captured in models if automated vehicles are to coexist successfully with human road users. Empirical studies of human road user behavior implicate a large number of underlying cognitive mechanisms, which taken together are well beyond the scope of existing computational models. Here, we note that for all of these putative mechanisms, computational theories exist in different subdisciplines of psychology, for more constrained tasks. We demonstrate how these separate theories can be generalized from abstract laboratory paradigms and integrated into a computational framework for modeling human road user interaction, combining Bayesian perception, a theory of mind regarding others’ intentions, behavioral game theory, long-term valuation of action alternatives, and evidence accumulation decision-making. We show that a model with these assumptions—but not simpler versions of the same model—can account for a number of previously unexplained phenomena in naturalistic driver–pedestrian road-crossing interactions, and successfully predicts interaction outcomes in an unseen data set. Our modeling results contribute to demonstrating the real-world value of the theories from which we draw, and address calls in psychology for cumulative theory-building, presenting human road use as a suitable setting for work of this nature. Our findings also underscore the formidable complexity of human interaction in road traffic, with strong implications for the requirements to set on development and testing of vehicle automation.

[1]  Yee Mun Lee,et al.  Who goes first? a distributed simulator study of vehicle-pedestrian interaction. , 2023, Accident; analysis and prevention.

[2]  Joshua E. Domeyer,et al.  Driver-Pedestrian Perceptual Models Demonstrate Coupling: Implications for Vehicle Automation , 2022, IEEE Transactions on Human-Machine Systems.

[3]  Hubert P. H. Shum,et al.  Interaction-Aware Decision-Making for Automated Vehicles Using Social Value Orientation , 2022, IEEE Transactions on Intelligent Vehicles.

[4]  Aravinda Ramakrishnan Srinivasan,et al.  Beyond RMSE: Do Machine-Learned Models of Road User Interaction Produce Human-Like Behavior? , 2022, IEEE Transactions on Intelligent Transportation Systems.

[5]  J. Dugdale,et al.  Agent-Based Modeling for Predicting Pedestrian Trajectories Around an Autonomous Vehicle , 2022, J. Artif. Intell. Res..

[6]  Gustav Markkula,et al.  A Utility Maximization Model of Pedestrian and Driver Interactions , 2021, IEEE Access.

[7]  Natasha Merat,et al.  Variable-Drift Diffusion Models of Pedestrian Road-Crossing Decisions , 2021, Computational Brain & Behavior.

[8]  A. Zgonnikov,et al.  A Human Factors Approach to Validating Driver Models for Interaction-aware Automated Vehicles , 2021, ACM Transactions on Human-Robot Interaction.

[9]  Jiquan Ngiam,et al.  Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[10]  Henry X. Liu,et al.  Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment , 2021, Nature Communications.

[11]  Raquel Urtasun,et al.  TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Yee Mun Lee,et al.  Learning to interpret novel eHMI: The effect of vehicle kinematics and eHMI familiarity on pedestrian' crossing behavior. , 2020, Journal of safety research.

[13]  Y. Niv,et al.  The case against economic values in the orbitofrontal cortex (or anywhere else in the brain). , 2020, Behavioral neuroscience.

[14]  Natasha Merat,et al.  Road users rarely use explicit communication when interacting in today’s traffic: implications for automated vehicles , 2020, Cognition, Technology & Work.

[15]  Matthias Althoff,et al.  Pedestrian Models for Autonomous Driving Part II: High-Level Models of Human Behavior , 2020, IEEE Transactions on Intelligent Transportation Systems.

[16]  Eiko I. Fried,et al.  Invisible Hands and Fine Calipers: A Call to Use Formal Theory as a Toolkit for Theory Construction , 2020, Perspectives on psychological science : a journal of the Association for Psychological Science.

[17]  J. Bärgman,et al.  Computational modeling of driver pre-crash brake response, with and without off-road glances: Parameterization using real-world crashes and near-crashes. , 2020, Accident; analysis and prevention.

[18]  Yee Mun Lee,et al.  Defining interactions: a conceptual framework for understanding interactive behaviour in human and automated road traffic , 2020, Theoretical Issues in Ergonomics Science.

[19]  L. Schulz,et al.  The Naïve Utility Calculus as a unified, quantitative framework for action understanding , 2019, Cognitive Psychology.

[20]  Javier Alonso-Mora,et al.  Social behavior for autonomous vehicles , 2019, Proceedings of the National Academy of Sciences.

[21]  Jay I. Myung,et al.  Global model analysis by parameter space partitioning. , 2019, Psychological review.

[22]  Guojie Ma,et al.  Drivers’ recognition of pedestrian road-crossing intentions: Performance and process , 2019, Transportation Research Part F: Traffic Psychology and Behaviour.

[23]  John D. Lee,et al.  Proxemics and Kinesics in Automated Vehicle–Pedestrian Communication: Representing Ethnographic Observations , 2019, Transportation Research Record: Journal of the Transportation Research Board.

[24]  Cynthia Breazeal,et al.  Machine behaviour , 2019, Nature.

[25]  Brandon M. Turner,et al.  Cognitive and Neural Bases of Multi-Attribute, Multi-Alternative, Value-based Decisions , 2019, Trends in Cognitive Sciences.

[26]  Stefania Bandini,et al.  Observation results on pedestrian-vehicle interactions at non-signalized intersections towards simulation , 2018, Transportation Research Part F: Traffic Psychology and Behaviour.

[27]  D. Navarro Between the Devil and the Deep Blue Sea: Tensions Between Scientific Judgement and Statistical Model Selection , 2018, Computational Brain & Behavior.

[28]  Anca D. Dragan,et al.  Planning for cars that coordinate with people: leveraging effects on human actions for planning and active information gathering over human internal state , 2018, Autonomous Robots.

[29]  Hans Straka,et al.  A New Perspective on Predictive Motor Signaling , 2018, Current Biology.

[30]  John K. Tsotsos,et al.  Understanding Pedestrian Behavior in Complex Traffic Scenes , 2018, IEEE Transactions on Intelligent Vehicles.

[31]  Malte Risto,et al.  Human-Vehicle Interfaces: The Power of Vehicle Movement Gestures in Human Road User Coordination , 2017 .

[32]  Natasha Merat,et al.  Sustained sensorimotor control as intermittent decisions about prediction errors: computational framework and application to ground vehicle steering , 2017, Biological Cybernetics.

[33]  Kevin Gurney,et al.  Multi-alternative decision-making with non-stationary inputs , 2016, Royal Society Open Science.

[34]  G. Pezzulo,et al.  Navigating the Affordance Landscape: Feedback Control as a Process Model of Behavior and Cognition , 2016, Trends in Cognitive Sciences.

[35]  Scott D. Brown,et al.  Diffusion Decision Model: Current Issues and History , 2016, Trends in Cognitive Sciences.

[36]  Meng Wang,et al.  Game theoretic approach for predictive lane-changing and car-following control , 2015 .

[37]  Simon F Giszter,et al.  Motor primitives—new data and future questions , 2015, Current Opinion in Neurobiology.

[38]  Duje Tadin,et al.  Unifying account of visual motion and position perception , 2015, Proceedings of the National Academy of Sciences.

[39]  Nicolas Marmaras,et al.  Drivers' communicative interactions: on-road observations and modelling for integration in future automation systems , 2014, Ergonomics.

[40]  P. Dayan,et al.  Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation , 2014, Cognitive, affective & behavioral neuroscience.

[41]  G. Pezzulo,et al.  Human Sensorimotor Communication: A Theory of Signaling in Online Social Interactions , 2013, PloS one.

[42]  Kevin Leyton-Brown,et al.  Predicting human behavior in unrepeated, simultaneous-move games , 2013, Games Econ. Behav..

[43]  Gregory C. DeAngelis,et al.  Bridging the gap between theories of sensory cue integration and the physiology of multisensory neurons , 2013, Nature Reviews Neuroscience.

[44]  Bingni W. Brunton,et al.  Rats and Humans Can Optimally Accumulate Evidence for Decision-Making , 2013, Science.

[45]  Andrew D. Wilson,et al.  Embodied Cognition is Not What you Think it is , 2013, Front. Psychology.

[46]  Dino J. Levy,et al.  The root of all value: a neural common currency for choice , 2012, Current Opinion in Neurobiology.

[47]  David W. Franklin,et al.  Computational Mechanisms of Sensorimotor Control , 2011, Neuron.

[48]  Ian David Loram,et al.  Intermittent control: a computational theory of human control , 2011, Biological Cybernetics.

[49]  Chris L. Baker,et al.  Action understanding as inverse planning , 2009, Cognition.

[50]  Dario D. Salvucci Rapid prototyping and evaluation of in-vehicle interfaces , 2009, TCHI.

[51]  Brett R Fajen,et al.  Perceptual learning and the visual control of braking , 2008, Perception & psychophysics.

[52]  Michelle M. Porter,et al.  Pedestrians' Normal Walking Speed and Speed When Crossing a Street , 2007 .

[53]  Dario D. Salvucci Modeling Driver Behavior in a Cognitive Architecture , 2006, Hum. Factors.

[54]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[55]  John R Anderson,et al.  An integrated theory of the mind. , 2004, Psychological review.

[56]  Serge P. Hoogendoorn,et al.  Simulation of pedestrian flows by optimal control and differential games , 2003 .

[57]  Zijiang J. He,et al.  Distance determined by the angular declination below the horizon , 2001, Nature.

[58]  James L. McClelland,et al.  The time course of perceptual choice: the leaky, competing accumulator model. , 2001, Psychological review.

[59]  A Várhelyi,et al.  Drivers' speed behaviour at a zebra crossing: a case study. , 1998, Accident; analysis and prevention.

[60]  R. Stein,et al.  Analysis of rapid stopping during human walking. , 1998, Journal of neurophysiology.

[61]  P. V. Lange,et al.  The impact of social value orientations on negotiator cognition and behavior , 1995 .

[62]  A. Thorstensson,et al.  Adaptations to changing speed in human locomotion: speed of transition between walking and running. , 1987, Acta physiologica Scandinavica.

[63]  Allen Newell,et al.  SOAR: An Architecture for General Intelligence , 1987, Artif. Intell..

[64]  Yee Mun Lee,et al.  Explaining unsafe pedestrian road crossing behaviours using a Psychophysics-based gap acceptance model , 2022, Safety Science.

[65]  Naveed Muhammad,et al.  A Survey on Motion Prediction of Pedestrians and Vehicles for Autonomous Driving , 2021, IEEE Access.

[66]  A. Gopnik,et al.  Natural theories of mind: Evolution, development and simulation of everyday mindreading , 2015 .

[67]  Rune Elvik,et al.  A review of game-theoretic models of road user behaviour. , 2014, Accident; analysis and prevention.

[68]  Michel Bierlaire,et al.  Specification, estimation and validation of a pedestrian walking behaviour model , 2007 .

[69]  H Pashler,et al.  How persuasive is a good fit? A comment on theory testing. , 2000, Psychological review.

[70]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[71]  Giovanni Pezzulo,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence Motor Simulation via Coupled Internal Models Using Sequential Monte Carlo , 2022 .