A Novel Virtual View Quality Enhancement Technique through a Learning of Synthesised Video

With the development of displaying techniques, free viewpoint video (FVV) system shows its potential to provide immersive perceptual feeling by changing viewpoints. To provide this luxury, a large number of high quality views have to be synthesised from limited number of viewpoints. However, in this process, a portion of the background is occluded by the foreground object in the generated synthesised videos. Recent techniques, i.e. view synthesized prediction using Gaussian model (VSPGM) and adaptive weighting between warped and learned foregrounds indicate that learning techniques may fill occluded areas almost correctly. However, these techniques use temporal correlation by assuming that original texture of the target viewpoint are already available to fill up occluded areas which is not a practical solution. Moreover, if a pixel position experiences foreground once during learning, the existing techniques considered it as foreground throughout the process. However, the actual fact is that after experiencing a foreground a pixel position can be background again. To address the aforementioned issues, in the proposed view synthesise technique, we apply Gaussian mixture modelling (GMM) on the output images of inverse mapping (IM) technique for further improving the quality of the synthesised videos. In this technique, the foreground and background pixel intensities are refined from adaptive weights of the output of inverse mapping and the pixel intensities from the corresponding model(s) of the GMM. This technique provides a better pixel correspondence, which improves 0.10~0.46dB PSNR compared to the IM technique.