Virtual reality technologies for the realistic simulation of excavators and construction machines: from VR-training simulators to telepresence systems

At the Institute of Robotics Research in Germany (IRF), an excavator- and construction machine simulator based on latest virtual reality technology has been developed. The main issues of the realization so far have been the real-time capability, the close-to-reality presentation of the environment and the physically correct simulation of the process, i.e. the simulation of the flow of the material handled with e.g. the simulated excavator. In the next step, it is being envisaged to enhance the system in a way that it cannot only be used for training, but also to command and supervise large construction machines in real world applications by means of virtual reality and automatic action planning components. Experiences gained from the control of space robots being controlled by methods of “projective virtual reality” will be introduced into this application to allow one driver to remotely control several excavators e.g. in a mining environment. In the paper, we describe how the simulation of the excavator and its interaction with the handled material can be handled mathematically and we will explain the basic ideas of how to “project” action that were carried in the virtual word onto physical excavators by employing the methods of projective virtual reality.