RCA: Ride Comfort-Aware Visual Navigation via Self-Supervised Learning

Under shared autonomy, wheelchair users expect vehicles to provide safe and comfortable rides while following users' high-level navigation plans. To find such a path, vehicles negotiate with different terrains and assess their traversal difficulty. Most prior works model surroundings either through geometric representations or semantic classifications, which do not reflect perceived motion intensity and ride comfort in downstream navigation tasks. We propose to model ride comfort explicitly in traversability analysis using proprioceptive sensing. We develop a self-supervised learning framework to predict traversability costmap from first-person-view images by leveraging vehicle states as training signals. Our approach estimates how the vehicle would “feel” if traversing over based on terrain appearances. We then show our navigation system provides human-preferred ride comfort through robot experiments together with a human evaluation study. The project could be found at https://sites.google.com/view/rca-navigation.

[1]  Howie Choset,et al.  Autonomous Exploration Development Environment and the Planning Algorithms , 2021, 2022 International Conference on Robotics and Automation (ICRA).

[2]  B. Siciliano,et al.  Autonomy in Physical Human-Robot Interaction: A Brief Survey , 2021, IEEE Robotics and Automation Letters.

[3]  P. B. Sujit,et al.  OFFSEG: A Semantic Segmentation Framework For Off-Road Driving , 2021, 2021 IEEE 17th International Conference on Automation Science and Engineering (CASE).

[4]  Xijun Zhao,et al.  Fine-Grained Off-Road Semantic Segmentation and Mapping via Contrastive Learning , 2021, 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Kris M. Kitani,et al.  Audio-Visual Self-Supervised Terrain Type Recognition for Ground Mobile Platforms , 2021, IEEE Access.

[6]  S. Levine,et al.  BADGR: An Autonomous Self-Supervised Learning-Based Navigation System , 2020, IEEE Robotics and Automation Letters.

[7]  Wolfram Burgard,et al.  Self-Supervised Visual Terrain Classification From Unsupervised Acoustic Feature Learning , 2019, IEEE Transactions on Robotics.

[8]  Lorenz Wellhausen,et al.  Learning quadrupedal locomotion over challenging terrain , 2020, Science Robotics.

[9]  Giovanni Muscato,et al.  Domain Adaptation for Outdoor Robot Traversability Estimation from RGB data with Safety-Preserving Loss , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[10]  Chen Hu,et al.  Falco: Fast likelihood‐based collision avoidance with extension to human‐guided navigation , 2020, J. Field Robotics.

[11]  H. Bülthoff,et al.  The role of acceleration and jerk in perception of above-threshold surge motion , 2020, Experimental Brain Research.

[12]  Huijing Zhao,et al.  Off-road Autonomous Vehicles Traversability Analysis and Trajectory Planning Based on Deep Inverse Reinforcement Learning , 2019, 2020 IEEE Intelligent Vehicles Symposium (IV).

[13]  Silvio Savarese,et al.  Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Tasks , 2019, IEEE Transactions on Robotics.

[14]  Krzysztof Walas,et al.  Where Should I Walk? Predicting Terrain Properties From Images Via Self-Supervised Learning , 2019, IEEE Robotics and Automation Letters.

[15]  Hang Zhang,et al.  Deep Texture Manifold for Ground Terrain Recognition , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[16]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[17]  Byron Boots,et al.  Agile Autonomous Driving using End-to-End Deep Imitation Learning , 2017, Robotics: Science and Systems.

[18]  Alessandro Giusti,et al.  Learning Ground Traversability From Simulations , 2017, IEEE Robotics and Automation Letters.

[19]  Beom Hee Lee,et al.  A Bayesian approach to terrain map inference based on vibration features , 2017, 2017 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI).

[20]  Wolfram Burgard,et al.  Deep spatiotemporal models for robust proprioceptive terrain classification , 2017, Int. J. Robotics Res..

[21]  Hung Manh La,et al.  A Comprehensive Review of Smart Wheelchairs: Past, Present, and Future , 2017, IEEE Transactions on Human-Machine Systems.

[22]  Osamu Mizuno,et al.  Wheelchair Users' Psychological Barrier Estimation Based on Inertial and Vital Data , 2016, HCI.

[23]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Takashi Kubota,et al.  Autonomous Terrain Classification With Co- and Self-Training Approach , 2016, IEEE Robotics and Automation Letters.

[25]  Roland Siegwart,et al.  Signed Distance Fields: A Natural Representation for Both Mapping and Planning , 2016 .

[26]  A. Koontz,et al.  Wheeled Mobility , 2015, BioMed research international.

[27]  Martial Hebert,et al.  Autonomy Infused Teleoperation with Application to BCI Manipulation , 2015, Robotics: Science and Systems.

[28]  Ji Zhang,et al.  LOAM: Lidar Odometry and Mapping in Real-time , 2014, Robotics: Science and Systems.

[29]  Karl Iagnemma,et al.  Self‐supervised terrain classification for planetary surface exploration rovers , 2012, J. Field Robotics.

[30]  James M. Rehg,et al.  Traversability classification using unsupervised on-line visual learning for outdoor robot navigation , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[31]  R P Gaal,et al.  Wheelchair rider injuries: causes and consequences for wheelchair design and selection. , 1997, Journal of rehabilitation research and development.