Supervised Learning Based Peripheral Vision System for Immersive Visual Experiences for Extended Display

Video display content can be extended to the walls of the living room around the TV using projection. The problem of providing appropriate projection content is hard for the computer and we solve this problem with deep neural network. We propose the peripheral vision system that provides the immersive visual experiences to the user by extending the video content using deep learning and projecting that content around the TV screen. The user may manually create the appropriate content for the existing TV screen, but it is too expensive to create it. The PCE (Pixel context encoder) network considers the center of the video frame as input and the outside area as output to extend the content using supervised learning. The proposed system is expected to pave a new road to the home appliance industry, transforming the living room into the new immersive experience platform.

[1]  Fabrice Urban,et al.  Extrafoveal Video Extension for an Immersive Viewing Experience , 2017, IEEE Transactions on Visualization and Computer Graphics.

[2]  Oliver Bimber,et al.  Embedded Entertainment with Smart Projectors , 2005, Computer.

[3]  Per Ola Kristensson,et al.  Breaking the Screen: Interaction Across Touchscreen Boundaries in Virtual Reality for Mobile Knowledge Workers , 2020, IEEE Transactions on Visualization and Computer Graphics.

[4]  Henry Fuchs,et al.  Manufacturing Application-Driven Foveated Near-Eye Displays , 2019, IEEE Transactions on Visualization and Computer Graphics.

[5]  Eric O. Postma,et al.  Light-weight pixel context encoders for image inpainting , 2018, ArXiv.

[6]  Hiroshi Ishikawa,et al.  Globally and locally consistent image completion , 2017, ACM Trans. Graph..

[7]  Hans-Peter Seidel,et al.  Luminance-contrast-aware foveated rendering , 2019, ACM Trans. Graph..

[8]  Yoav Y. Schechner,et al.  Ultrawide Foveated Video Extrapolation , 2011, IEEE Journal of Selected Topics in Signal Processing.

[9]  Tobias Ritschel,et al.  Perceptual rasterization for head-mounted display image synthesis , 2018, ACM Trans. Graph..

[10]  Oliver Bimber,et al.  Superimposing Pictorial Artwork with Projected Imagery , 2005, IEEE Multim..

[11]  Katerina Mania,et al.  Near‐Eye Display and Tracking Technologies for Virtual and Augmented Reality , 2019, Comput. Graph. Forum.