AppGAN: Generative Adversarial Networks for Generating Robot Approach Behaviors into Small Groups of People

Robots that navigate to approach free-standing conversational groups should do so in a safe and socially acceptable manner. This is challenging since it not only requires the robot to plot trajectories that avoid collisions with members of the group, but also to do so without making those in the group feel uncomfortable, for example, by moving too close to them or approaching them from behind. Previous trajectory prediction models focus primarily on formations of walking pedestrians, and those models that do consider approach behaviours into free-standing conversational groups typically have handcrafted features and are only evaluated via simulation methods, limiting their effectiveness. In this paper, we propose AppGAN, a novel trajectory prediction model capable of generating trajectories into free-standing conversational groups trained on a dataset of safe and socially acceptable paths. We evaluate the performance of our model with state-of-the-art trajectory prediction methods on a semi-synthetic dataset. We show that our model outperforms baselines by taking advantage of the GAN framework and our novel group interaction module.

[1]  E. Hall,et al.  The Hidden Dimension , 1970 .

[2]  Xiaogang Wang,et al.  Understanding pedestrian behaviors from stationary crowd groups , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Stefania Bandini,et al.  Towards an Integrated Approach to Crowd Analysis and Crowd Synthesis: a Case Study and First Results , 2013, Pattern Recognit. Lett..

[4]  S. Dai,et al.  Centrifugal force model for pedestrian dynamics. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[5]  Hannes Högni Vilhjálmsson,et al.  Social Perception and Steering for Online Avatars , 2008, IVA.

[6]  Kerstin Dautenhahn,et al.  Robotic etiquette: Results from user studies involving a fetch and carry task , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[7]  P. Molnár Social Force Model for Pedestrian Dynamics Typeset Using Revt E X 1 , 1995 .

[8]  Takayuki Kanda,et al.  Navigation for human-robot interaction tasks , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[9]  Silvio Savarese,et al.  Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  A. Kendon Conducting Interaction: Patterns of Behavior in Focused Encounters , 1990 .

[11]  Wei Xu,et al.  Video Paragraph Captioning Using Hierarchical Recurrent Neural Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Masayuki Nakajima,et al.  Personal Space Modeling for Human-Computer Interaction , 2009, ICEC.

[13]  Adrien Treuille,et al.  Continuum crowds , 2006, ACM Trans. Graph..

[14]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[15]  David Rye,et al.  How should a robot approach two people? , 2017, HRI 2017.

[16]  Maja J. Mataric,et al.  Proxemics and performance: Subjective human evaluations of autonomous sociable robot distance and social signal understanding , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  Nicu Sebe,et al.  Analyzing Free-standing Conversational Groups: A Multimodal Approach , 2015, ACM Multimedia.

[18]  Rachid Alami,et al.  Human-aware robot navigation: A survey , 2013, Robotics Auton. Syst..

[19]  Dinesh Manocha,et al.  Reciprocal Velocity Obstacles for real-time multi-agent navigation , 2008, 2008 IEEE International Conference on Robotics and Automation.

[20]  Silvio Savarese,et al.  SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Xuan-Tung Truong,et al.  “To Approach Humans?”: A Unified Framework for Approaching Pose Prediction and Socially Aware Robot Navigation , 2018, IEEE Transactions on Cognitive and Developmental Systems.

[22]  Javier V. Gómez,et al.  Fast marching solution for the social path planning problem , 2014, ICRA.

[23]  J. Sethian,et al.  Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations , 1988 .

[24]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Rahul Narain,et al.  Aggregate dynamics for dense crowd simulation , 2009, SIGGRAPH 2009.

[26]  Sridha Sridharan,et al.  Soft + Hardwired Attention: An LSTM Framework for Human Trajectory Prediction and Abnormal Event Detection , 2017, Neural Networks.

[27]  R. Hughes The flow of human crowds , 2003 .

[28]  Martin Buss,et al.  Proactively Approaching Pedestrians with an Autonomous Mobile Robot in Urban Environments , 2012, ISER.

[29]  Elena Torta,et al.  Design of a Parametric Model of Personal Space for Robotic Social Navigation , 2013, Int. J. Soc. Robotics.

[30]  Vishnu K. Narayanan,et al.  On equitably approaching and joining a group of interacting humans , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[31]  Kerstin Dautenhahn,et al.  Social Roles and Baseline Proxemic Preferences for a Domestic Service Robot , 2014, Int. J. Soc. Robotics.

[32]  Hannes Högni Vilhjálmsson,et al.  Study of Nine People in a Hallway: Some Simulation Challenges , 2018, IVA.

[33]  Francesco Setti,et al.  F-Formation Detection: Individuating Free-Standing Conversational Groups in Images , 2015, PloS one.

[34]  Sridha Sridharan,et al.  GD-GAN: Generative Adversarial Networks for Trajectory Prediction and Group Detection in Crowds , 2018, ACCV.

[35]  John R. Hershey,et al.  Attention-Based Multimodal Fusion for Video Description , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[36]  Scott E. Hudson,et al.  Towards Robot Autonomy in Group Conversations: Understanding the Effects of Body Orientation and Gaze , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[37]  Illah R. Nourbakhsh,et al.  A survey of socially interactive robots , 2003, Robotics Auton. Syst..

[38]  S. Osher,et al.  Algorithms Based on Hamilton-Jacobi Formulations , 1988 .

[39]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[40]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[41]  Daniel Cremers,et al.  SPENCER: A Socially Aware Service Robot for Passenger Guidance and Help in Busy Airports , 2015, FSR.

[42]  Subramanian Ramanathan,et al.  Uncovering Interactions and Interactors: Joint Estimation of Head, Body Orientation and F-Formations from Surveillance Videos , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[43]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[44]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[45]  Jean Oh,et al.  Social Attention: Modeling Attention in Human Crowds , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).