Extrafoveal Video Extension for an Immersive Viewing Experience

Between the recent popularity of virtual reality (VR) and the development of 3D, immersion has become an integral part of entertainment concepts. Head-mounted Display (HMD) devices are often used to afford users a feeling of immersion in the environment. Another technique is to project additional material surrounding the viewer, as is achieved using cave systems. As a continuation of this technique, it could be interesting to extend surrounding projection to current television or cinema screens. The idea would be to entirely fill the viewer<sc>'</sc>s field of vision, thus providing them with a more complete feeling of being in the scene and part of the story. The appropriate content can be captured using large field of view (FoV) technology, using a rig of cameras for 110<inline-formula> <tex-math notation="LaTeX">$^{\circ}$</tex-math><alternatives><inline-graphic xlink:href="guillotel-ieq1-2527649.gif"/> </alternatives></inline-formula> to 360<inline-formula><tex-math notation="LaTeX">$^{\circ}$</tex-math><alternatives> <inline-graphic xlink:href="guillotel-ieq2-2527649.gif"/></alternatives></inline-formula> capture, or created using computer-generated images. The FoV is, however, rather limited in its use for existing (legacy) content, achieving between 36 to 90 degrees (<inline-formula><tex-math notation="LaTeX">$^{\circ}$</tex-math><alternatives> <inline-graphic xlink:href="guillotel-ieq3-2527649.gif"/></alternatives></inline-formula>) field, depending on the distance from the screen. This paper seeks to improve this FoV limitation by proposing computer vision techniques to extend such legacy content to the peripheral (extrafoveal) vision without changing the original creative intent or damaging the viewer<sc>'</sc>s experience. A new methodology is also proposed for performing user tests in order to evaluate the quality of the experience and confirm that the sense of immersion has been increased. This paper thus presents: i) an algorithm to spatially extend the video based on human vision characteristics, ii) its subjective results compared to state-of-the-art techniques, iii) the protocol required to evaluate the quality of the experience (QoE), and iv) the results of the user tests.