Render: Predicting Scenes

The main goal of scene prediction is to generate an image of a virtual world which can be directly compared to an image from a real sensor. How accurate the prediction must be in order for the two images to be comparable depends upon the specific application domain and the comparison metric used. Intuitively, the closer the prediction mirrors reality, the higher the chance the RMR (Render-Match-Refine) algorithm will be successful. Unfortunately, generating a perfect prediction is an impossible task. Information such as the background scene context, lighting, and complex camera distortions are difficult to model and often cannot be inferred from a static sensor image. Therefore, the role of prediction is to produce an image, at the resolution of the real sensor, that provides information exploitable by the subsequent stages of the RMR method (matching and refinement).