Viewpoint Quality Evaluation for Augmented Virtual Environment

Augmented Virtual Environment (AVE) fuses real-time video streaming with virtual scenes to provide a new capability of the real-world run-time perception. Although this technique has been developed for many years, it still suffers from the fusion correctness, complexity and the image distortion during flying. The image distortion could be commonly found in an AVE system, which is decided by the viewpoint of the environment. Existing work lacks of the evaluation of the viewpoint quality, and then failed to optimize the fly path for AVE. In this paper, we propose a novel method of viewpoint quality evaluation (VQE), taking texture distortion as evaluation metric. The texture stretch and object fragment are taken as the main factors of distortion. We visually compare our method with viewpoint entropy on campus scene, demonstrating that our method is superior in reflecting distortion degree. Furthermore, we conduct a user study, revealing that our method is suitable for the good quality demonstration with viewpoint control for AVE.

[1]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[2]  Guosheng Lin,et al.  Deep convolutional neural fields for depth estimation from a single image , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Zhong Zhou,et al.  Path Planning for Virtual-Reality Integration Surveillance System , 2018 .

[4]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[5]  Ian D. Reid,et al.  RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Ramesh C. Jain,et al.  Immersive video , 1996, Proceedings of the IEEE 1996 Virtual Reality Annual International Symposium.

[7]  Hao Jiang,et al.  Big Earth Data: a new challenge and opportunity for Digital Earth’s development , 2017, Int. J. Digit. Earth.

[8]  Yi Yang,et al.  Attention to Scale: Scale-Aware Semantic Image Segmentation , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Song-Chun Zhu,et al.  Image Parsing with Stochastic Scene Grammar , 2011, NIPS.

[10]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Mateu Sbert,et al.  Viewpoint Selection using Viewpoint Entropy , 2001, VMV.

[12]  Changhai Xu,et al.  Real-time indoor scene understanding using Bayesian filtering with motion cues , 2011, 2011 International Conference on Computer Vision.

[13]  David W. Jacobs,et al.  Mesh saliency , 2005, SIGGRAPH 2005.

[14]  Suya You,et al.  Augmented virtual environments (AVE): dynamic fusion of imagery and 3D models , 2003, IEEE Virtual Reality, 2003. Proceedings..

[15]  Pere-Pau Vázquez,et al.  Automatic view selection through depth-based view stability analysis , 2009, The Visual Computer.

[16]  Yu Hu,et al.  GeoCueDepth: Exploiting geometric structure cues to estimate depth from a single image , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  Mongi A. Abidi,et al.  Shape analysis algorithm based on information theory , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[18]  Dimitri Plemenos,et al.  Viewpoint Quality: Measures and Applications , 2005, CAe.

[19]  Sinisa Todorovic,et al.  Monocular Depth Estimation Using Neural Regression Forest , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Hans-Peter Seidel,et al.  Towards Stable and Salient Multi-View Representation of 3D Shapes , 2006, IEEE International Conference on Shape Modeling and Applications 2006 (SMI'06).

[21]  Andrea Bönsch,et al.  Comparison and Evaluation of Viewpoint Quality Estimation Algorithms for Immersive Virtual Environments , 2015, ICAT-EGVE.

[22]  Marc Christie,et al.  A Semantic Space Partitioning Approach to Virtual Camera Composition , 2005, Comput. Graph. Forum.

[23]  Xuming He,et al.  Discrete-Continuous Depth Estimation from a Single Image , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Oisin Mac Aodha,et al.  Unsupervised Monocular Depth Estimation with Left-Right Consistency , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Silvia Biasotti,et al.  What’s in an image? , 2005, The Visual Computer.

[26]  Deb Roy,et al.  An immersive system for browsing and visualizing surveillance video , 2010, ACM Multimedia.

[27]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Xiangtao Fan,et al.  Augmented virtual environment: fusion of real-time video and 3D models in the digital earth system , 2017, Int. J. Digit. Earth.

[29]  Suya You,et al.  3D video surveillance with Augmented Virtual Environments , 2003, IWVS '03.

[30]  Takeo Kanade,et al.  Geometric reasoning for single image structure recovery , 2009, CVPR.