Prediction of Human Trajectory Following a Haptic Robotic Guide Using Recurrent Neural Networks

Social intelligence is an important requirement for enabling robots to collaborate with people. In particular, human path prediction is an essential capability for robots in that it prevents potential collision with a human and allows the robot to safely make larger movements. In this paper, we present a method for predicting the trajectory of a human who follows a haptic robotic guide without using sight, which is valuable for assistive robots that aid the visually impaired. We apply a deep learning method based on recurrent neural networks using multimodal data: (1) human trajectory, (2) movement of the robotic guide, (3) haptic input data measured from the physical interaction between the human and the robot, (4) human depth data. We collected actual human trajectory and multimodal response data through indoor experiments. Our model outperformed the baseline result while using only the robot data with the observed human trajectory, and it shows even better results when using additional haptic and depth data.

[1]  Woohyun Kim,et al.  Observation of Human Trajectory in Response to Haptic Feedback from Mobile Robot , 2018, 2018 18th International Conference on Control, Automation and Systems (ICCAS).

[2]  Domenico Prattichizzo,et al.  Cooperative Navigation for Mixed Human–Robot Teams Using Haptic Feedback , 2017, IEEE Transactions on Human-Machine Systems.

[3]  Jongsoo Baek,et al.  Effect of Redundant Haptic Information on Task Performance during Visuo-Tactile Task Interruption and Recovery , 2016, Front. Psychol..

[4]  Chun-Hsu Ko,et al.  Motion Guidance for a Passive Robot Walking Helper via User's Applied Hand Forces , 2016, IEEE Transactions on Human-Machine Systems.

[5]  Seung-Chan Kim,et al.  Haptic interaction with virtual geometry on robotic touch surface , 2014, SIGGRAPH ASIA Emerging Technologies.

[6]  Mark Reynolds,et al.  SS-LSTM: A Hierarchical LSTM Model for Pedestrian Trajectory Prediction , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[7]  Michael J. Black,et al.  On Human Motion Prediction Using Recurrent Neural Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[9]  Lyuba Alboul,et al.  Following a Robot using a Haptic Interface without Visual Feedback , 2014, ACHI 2014.

[10]  Jitendra Malik,et al.  Recurrent Network Models for Human Dynamics , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[11]  Yiannis Demiris,et al.  Learning Shared Control by Demonstration for Personalized Wheelchair Assistance , 2018, IEEE Transactions on Haptics.

[12]  Kerstin Dautenhahn,et al.  Socially intelligent robots: dimensions of human–robot interaction , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[13]  Wanjoo Park,et al.  Combining Full and Partial Haptic Guidance Improves Handwriting Skills Development , 2018, IEEE Transactions on Haptics.

[14]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[15]  Silvio Savarese,et al.  Social LSTM: Human Trajectory Prediction in Crowded Spaces , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Christopher D. Wickens,et al.  Multiple resources and performance prediction , 2002 .

[17]  David A. Winter,et al.  Human balance and posture control during standing and walking , 1995 .

[18]  Cagatay Basdogan,et al.  Conveying intentions through haptics in human-computer collaboration , 2011, 2011 IEEE World Haptics Conference.

[19]  Mary-Anne Williams,et al.  Robot Social Intelligence , 2012, ICSR.

[20]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[21]  Sandra Hirche,et al.  An experience-driven robotic assistant acquiring human knowledge to improve haptic cooperation , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[22]  Jiwon Seo,et al.  Observation of Human Response to a Robotic Guide Using a Variational Autoencoder , 2019, 2019 Third IEEE International Conference on Robotic Computing (IRC).

[23]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[24]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[25]  Gita Alaghband,et al.  Scene-LSTM: A Model for Human Trajectory Prediction , 2018, ArXiv.

[26]  Uwe D. Hanebeck,et al.  Wide-area haptic guidance: Taking the user by the hand , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.