Photo-realistic scene prediction of partially unknown environments for the compensation of time delays in telepresence applications

Predictive displays have proven their suitability to compensate time delays in visual feedback which impair the performance of teleoperation applications. Most current implementations employ simple computer graphics for the prediction of the visual feedback given to the operator who is controlling the remote manipulation. Furthermore, they require a well-known remote environment. The aim of the presented work is to improve the efficacy of the predictive display by increasing the degree of immersion and by enhancing the system's flexibility. Potential applications include maintenance and repair of machines located in remote plant- or office-type environments, which are typically non-static and a priori only partially known. Communication media are high- and medium-bandwidth LANs and WANs inducing round trip delays varying from 0.2 to 2 seconds. Additionally, a sufficient bandwidth cannot be assured during the entire remote operation. The paper describes a predictive display system which achieves photo-realism based on autonomously explored and updated scene models. Model acquisition is performed with a binocular CCD camera system. Photo-realism is obtained by using computer graphics techniques, especially the mapping of original, real textures. Besides the concepts, an early stage of implementation and first experimental results in scene reconstruction and photo-realistic scene prediction are presented.