Visual Quality of 3D Meshes With Diffuse Colors in Virtual Reality: Subjective and Objective Evaluation

Surface meshes associated with diffuse color attributes are becoming popular multimedia contents. They provide a high degree of realism and allow 6DoF interactions in immersive environments. Just like other types of multimedia, 3D meshes are subject to a wide range of processing, which result in a loss of quality of the final rendered scene. Thus, both subjective studies and objective metrics are needed to understand and predict this visual loss. In this work, we introduce a large dataset of 480 animated meshes with diffuse color information, and associated with perceived quality judgments. The stimuli were generated from 5 source models subjected to geometry and color distortions. Each stimulus was associated with 6 hypothetical rendering trajectories: combinations of 3 viewpoints and 2 animations. A total of 11520 quality judgments (24 per stimulus) were acquired in a subjective experiment conducted in virtual reality. The results allowed us to explore the influence of source models, animations and viewpoints on both quality scores and their confidence intervals. Based on these findings, we propose the first metric for quality assessment of 3D meshes with diffuse colors, which works entirely on the mesh domain. This metric incorporates perceptually-relevant curvature-based and color-based features. We evaluate its performance, as well as several Image Quality Metrics (IQMs), on two datasets: ours and a dataset of distorted textured meshes. Our metric demonstrates good results and better stability than IQMs. Finally, we investigated how the knowledge of the viewpoint (i.e.,the visible parts of the 3D model) may improve objective metrics results.

[1]  Rafal Mantiuk,et al.  Quality Assessment in Computer Graphics , 2015 .

[2]  Rajiv Soundararajan,et al.  Study of Subjective and Objective Quality Assessment of Video , 2010, IEEE Transactions on Image Processing.

[3]  Irene Cheng,et al.  Quality metric for approximating subjective evaluation of 3-D objects , 2005, IEEE Transactions on Multimedia.

[4]  Longin Jan Latecki,et al.  No-reference mesh visual quality assessment via ensemble of convolutional neural networks and compact multi-linear pooling , 2020, Pattern Recognit..

[5]  Hans-Peter Seidel,et al.  Dataset and Metrics for Predicting Local Visible Differences , 2018, ACM Trans. Graph..

[6]  Karel Fliegel,et al.  On the accuracy of objective image and video quality models: New methodology for performance evaluation , 2016, 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX).

[7]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[8]  Catarina Brites,et al.  Point Cloud Rendering After Coding: Impacts on Subjective and Objective Quality , 2021, IEEE Transactions on Multimedia.

[9]  Philipp Urban,et al.  Color-Image Quality Assessment: From Prediction to Optimization , 2014, IEEE Transactions on Image Processing.

[10]  Guillaume Lavoué,et al.  A Multiscale Metric for 3D Mesh Visual Quality Assessment , 2011, Comput. Graph. Forum.

[11]  Touradj Ebrahimi,et al.  A novel methodology for quality assessment of voxelized point clouds , 2018, Optical Engineering + Applications.

[12]  Adam Finkelstein,et al.  A no-reference metric for evaluating the quality of motion deblurring , 2013, ACM Trans. Graph..

[13]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[14]  A. Bovik,et al.  Study of 3D Virtual Reality Picture Quality , 2019, IEEE Journal of Selected Topics in Signal Processing.

[15]  Ingmar Lissner,et al.  Toward a Unified Color Space for Perception-Based Image Processing , 2012, IEEE Transactions on Image Processing.

[16]  Guillaume Lavoué,et al.  PCQM: A Full-Reference Quality Metric for Colored 3D Point Clouds , 2020, 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX).

[17]  Kai Wang,et al.  A Curvature Tensor Distance for Mesh Visual Quality Assessment , 2012, ICCVG.

[18]  Ricardo L. de Queiroz,et al.  A comprehensive study of the rate-distortion performance in MPEG point cloud compression , 2019, APSIPA Transactions on Signal and Information Processing.

[19]  Zhi Li,et al.  Recover Subjective Quality Scores from Noisy Measurements , 2016, 2017 Data Compression Conference (DCC).

[20]  Beatriz Sousa Santos,et al.  A Perceptual Data Repository for Polygonal Meshes , 2009, 2009 Second International Conference in Visualisation.

[21]  Ingmar Lissner,et al.  Image-Difference Prediction: From Grayscale to Color , 2013, IEEE Transactions on Image Processing.

[22]  Sangwook Lee,et al.  Comparison of subjective video quality assessment methods for multimedia applications , 2007 .

[23]  Kai Wang,et al.  Just Noticeable Distortion Profile for Flat-Shaded 3D Mesh Surfaces , 2016, IEEE Transactions on Visualization and Computer Graphics.

[24]  Irene Cheng,et al.  Subjective and Objective Visual Quality Assessment of Textured 3D Meshes , 2016, ACM Trans. Appl. Percept..

[25]  Tolga K. Çapin,et al.  A machine learning framework for full-reference 3D shape quality assessment , 2018, The Visual Computer.

[26]  Michael Garland,et al.  Surface simplification using quadric error metrics , 1997, SIGGRAPH.

[27]  Guillaume Lavoué,et al.  Rate-distortion optimization for progressive compression of 3D mesh with color attributes , 2011, The Visual Computer.

[28]  Libor Vása,et al.  Perceptual Metrics for Static and Dynamic Triangle Meshes , 2013, Eurographics.

[29]  Libor Vása,et al.  On the Efficiency of Image Metrics for Evaluating the Visual Quality of 3D Models , 2016, IEEE Transactions on Visualization and Computer Graphics.

[30]  Guillaume Lavoué,et al.  A local roughness measure for 3D meshes and its application to visual masking , 2009, TAP.

[31]  Petros Daras,et al.  Subjective Visual Quality Assessment of Immersive 3D Media Compressed by Open-Source Static 3D Mesh Codecs , 2019, MMM.

[32]  Atilla Baskurt,et al.  Evaluating the local visibility of geometric artifacts , 2015, SAP.

[33]  C.-C. Jay Kuo,et al.  Optimized mesh and texture multiplexing for progressive textured model transmission , 2004, MULTIMEDIA '04.

[34]  Libor Vása,et al.  Dihedral Angle Mesh Error: a fast perception correlated distortion measure for fixed connectivity triangle meshes , 2012, Comput. Graph. Forum.

[35]  Wolfgang Heidrich,et al.  HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions , 2011, ACM Trans. Graph..

[36]  T. Ebrahimi,et al.  Watermarked 3-D Mesh Quality Assessment , 2007, IEEE Transactions on Multimedia.

[37]  Narciso García,et al.  Subjective Assessment of Adaptive Media Playout for Video Streaming , 2019, 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX).

[38]  Patrick Le Callet,et al.  Comparison of subjective methods, with and without explicit reference, for quality assessment of 3D graphics , 2019, SAP.

[39]  Zhengfang Duanmu,et al.  Perceptual Quality Assessment of 3d Point Clouds , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[40]  Johann Schrammel,et al.  VRate: A Unity3D Asset for integrating Subjective Assessment Questionnaires in Virtual Environments , 2018, 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX).

[41]  Pan Gao,et al.  Textured Mesh vs Coloured Point Cloud: A Subjective Study for Volumetric Video Compression , 2020, 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX).

[42]  Bernice E. Rogowitz,et al.  Are image quality metrics adequate to evaluate the quality of geometric objects? , 2001, IS&T/SPIE Electronic Imaging.

[43]  Touradj Ebrahimi,et al.  Perceptually driven 3D distance metrics with application to watermarking , 2006, SPIE Optics + Photonics.

[44]  Guillaume Lavoué,et al.  Progressive compression of arbitrary textured meshes , 2016, Comput. Graph. Forum.

[45]  Ghassan Al-Regib,et al.  BaTex3: Bit Allocation for Progressive Transmission of Textured 3-D Models , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[46]  Benjamin Watson,et al.  Measuring and predicting visual fidelity , 2001, SIGGRAPH.

[47]  Kai Wang,et al.  A fast roughness-based approach to the assessment of 3D mesh visual quality , 2012, Comput. Graph..

[48]  Pierre Alliez,et al.  Anisotropic polygonal remeshing , 2003, ACM Trans. Graph..

[49]  Touradj Ebrahimi,et al.  Point cloud quality evaluation: Towards a definition for test conditions , 2019, 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX).

[50]  Basile Sauvage,et al.  Visual Quality Assessment of 3D Models , 2017, ACM Trans. Appl. Percept..

[51]  Ke Gu,et al.  Prediction of the Influence of Navigation Scan-Path on Perceived Quality of Free-Viewpoint Videos , 2018, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[52]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .