Predictive safety based on track-before-detect for teleoperated driving through communication time delay

Teleoperated driving is known as a transient technology toward full autonomous driving in urban areas. However, this mobility concept suffers mainly from the communication time delay, which may result in safety hazards as well as stop-and-go driving behavior in crowded inner-city areas. This paper presents a novel active safety concept to assist the human operator of the teleoperated vehicle considering the communication time delay. The proposed system reacts not only to the actual driving hazards, but also to the upcoming hazards the human operator is not aware of because of time delay. For this purpose, it predicts the future trajectories of dynamic objects in the vehicle surroundings using a stereo vision based track-before-detect approach and reacts autonomously to the predicted hazards through speed control. After each intervention, the human operator is informed about the autonomous intervention of the vehicle by a Human-Machine-Interface (HMI), having the ability to override this intervention. Results of the test drives show an overall increase of the safety by reduction of Time-To-Collision as well as an improvement of the acceptance of teleoperated driving through the reduction of the overall triggered deceleration during driving in urban areas.

[1]  Javier Sánchez Pérez,et al.  TV-L1 Optical Flow Estimation , 2013, Image Process. Line.

[2]  Markus Lienkamp,et al.  Prediction Methods for Teleoperated Road Vehicles , 2013 .

[3]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[4]  Uwe Franke,et al.  Dense, Robust, and Accurate Motion Field Estimation from Stereo Image Sequences in Real-Time , 2010, ECCV.

[5]  Horst Bischof,et al.  A Duality Based Approach for Realtime TV-L1 Optical Flow , 2007, DAGM-Symposium.

[6]  Clemens Rabe,et al.  Detection of moving objects by spatio-temporal motion analysis , 2012 .

[7]  Thomas B. Sheridan,et al.  Space teleoperation through time delay: review and prognosis , 1993, IEEE Trans. Robotics Autom..

[8]  Markus Maurer,et al.  Forward Collision Warning and Avoidance , 2012 .

[9]  S. Buzzi,et al.  Track-before-detect procedures for early detection of moving target from airborne radars , 2005, IEEE Transactions on Aerospace and Electronic Systems.

[10]  Dariu Gavrila,et al.  Active Pedestrian Safety by Automatic Braking and Evasive Steering , 2011, IEEE Transactions on Intelligent Transportation Systems.

[11]  Margaret Lucas,et al.  What is Mechanical Engineering , 2011 .

[12]  Hermann Winner,et al.  Handbook of Driver Assistance Systems: Basic Information, Components and Systems for Active Safety and Comfort , 2015 .

[13]  M M Minderhoud,et al.  Extended time-to-collision measures for road traffic safety assessment. , 2001, Accident; analysis and prevention.

[14]  Tito Tang,et al.  Teleoperiertes Fahren – Sichere und robuste Datenverbindungen , 2014 .

[15]  Markus Lienkamp,et al.  Human-machine interaction as key technology for driverless driving - A trajectory-based shared autonomy control approach , 2012, 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication.

[16]  K. Salisbury,et al.  Issues in human/computer control of dexterous remote hands , 1988 .

[17]  Julius Ziegler,et al.  Making Bertha Drive—An Autonomous Journey on a Historic Route , 2014, IEEE Intelligent Transportation Systems Magazine.

[18]  Uwe Franke,et al.  6D-Vision: Fusion of Stereo and Motion for Robust Environment Perception , 2005, DAGM-Symposium.

[19]  Julius Ziegler,et al.  StereoScan: Dense 3d reconstruction in real-time , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[20]  Charles E. Thorpe,et al.  Collaborative control: a robot-centric model for vehicle teleoperation , 2001 .

[21]  Markus Lienkamp,et al.  Interactive path planning for teleoperated road vehicles in urban environments , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[22]  Markus Lienkamp,et al.  A System Design for Teleoperated Road Vehicles , 2013, ICINCO.

[23]  Hermann Winner,et al.  Fundamentals of Collision Protection Systems , 2015 .

[24]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..