An Adaptable and Immersive Real Time Interface for Resolving System Limitations of Automated Vehicles with Teleoperation

Due to the challenges of autonomous driving backup options like teleoperation become a relevant solution for critical scenarios an automated vehicle might face. To enable teleoperated systems two main problems have to be solved: Safely controlling the vehicle under latency, and presenting the sensor data from the vehicle to the operator in such a way, that the operator can easily understand the vehicle’s environment and the vehicles current state. The focus of this paper is solving the second problem and therefore the development of a novel human-machine-interface for the teleoperation of automated vehicles. For the development, the human-centered-design process is used, in which the whole teleoperation system is analyzed in the context of autonomous driving and requirements for the new interface are derived. Based on those requirements the new concept is designed and implemented. Finally, first results from real driving scenarios and novel features, like adaptive camera projection or scene configuration, are presented.