Joint effects of depth-aiding augmentations and viewing positions on the quality of experience in augmented telepresence

Virtual and augmented reality is increasingly prevalent in industrial applications, such as remote control of industrial machinery, due to recent advances in head-mounted display technologies and low-latency communications via 5G. However, the influence of augmentations and camera placement-based viewing positions on operator performance in telepresence systems remains unknown. In this paper, we investigate the joint effects of depth-aiding augmentations and viewing positions on the quality of experience for operators in augmented telepresence systems. A study was conducted with 27 non-expert participants using a real-time augmented telepresence system to perform a remote-controlled navigation and positioning task, with varied depth-aiding augmentations and viewing positions. The resulting quality of experience was analyzed via Likert opinion scales, task performance measurements, and simulator sickness evaluation. Results suggest that reducing the reliance on stereoscopic depth perception via camera placement has a significant benefit to operator performance and quality of experience. Conversely, the depth-aiding augmentations can partly mitigate the negative effects of inferior viewing positions. However the viewing-position based monoscopic and stereoscopic depth cues tend to dominate over cues based on augmentations. There is also a discrepancy between the participants’ subjective opinions on augmentation helpfulness, and its observed effects on positioning task performance.

[1]  Robert S. Kennedy,et al.  Simulator Sickness Questionnaire: An enhanced method for quantifying simulator sickness. , 1993 .

[2]  Fabrizio Cutolo,et al.  Perspective Preserving Solution for Quasi-Orthoscopic Video See-Through HMDs , 2018 .

[3]  Gregor Geršak,et al.  Effect of VR technology matureness on VR sickness , 2018, Multimedia Tools and Applications.

[4]  Zhi Zhang,et al.  LIFE: A FLEXIBLE TESTBED FOR LIGHT FIELD EVALUATION , 2018, 2018 - 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).

[5]  Jeanine K Stefanucci,et al.  Affordances as a Measure of Perceptual Fidelity in Augmented Reality , 2018 .

[6]  J. Edward Swan,et al.  Matching and Reaching Depth Judgments with Real and Augmented Reality Targets , 2015, IEEE Transactions on Visualization and Computer Graphics.

[7]  Bernhard E. Riecke,et al.  Comparing input methods and cursors for 3D positioning with head-mounted displays , 2018, SAP.

[8]  Pablo Pérez,et al.  MIRO360: A Tool for Subjective Assessment of 360 Degree Video for ITU-T P.360-VR , 2019, 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX).

[9]  Ina Bornkessel-Schlesewsky,et al.  A Comparison of Predictive Spatial Augmented Reality Cues for Procedural Tasks , 2018, IEEE Transactions on Visualization and Computer Graphics.

[10]  Michael Finke,et al.  Assessment of Risks and Benefits of Context-Adaptive Augmented Reality for Aerodrome Control Towers , 2018, 2018 IEEE/AIAA 37th Digital Avionics Systems Conference (DASC).

[11]  Alexander Raake,et al.  Quality of Experience: Advanced Concepts, Applications and Methods , 2014 .

[12]  Marc Hassenzahl,et al.  User experience - a research agenda , 2006, Behav. Inf. Technol..

[13]  Yongtian Wang,et al.  Employing Different Viewpoints for Remote Guidance in a Collaborative Augmented Environment , 2018, CCHI.

[14]  Truong Cong Thang,et al.  A subjective study on QoE of 360 video for VR communication , 2017, 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP).

[15]  Carlo Alberto Avizzano,et al.  Augmented reality-aided tele-presence system for robot manipulation in industrial manufacturing , 2015, VRST.

[16]  Markus Fiedler,et al.  Quality of Experience: From Assessment to Application (Dagstuhl Seminar 15022) , 2015, Dagstuhl Reports.

[17]  Shojiro Nagata,et al.  How to reinforce perception of depth in single two-dimensional pictures , 1991 .

[18]  S. Leung,et al.  Can Likert Scales be Treated as Interval Scales?—A Simulation Study , 2017 .

[19]  Sebastian Möller,et al.  QoE beyond the MOS: an in-depth look at QoE via better metrics and their relation to MOS , 2016, Quality and User Experience.

[20]  Peter Schelkens,et al.  Qualinet White Paper on Definitions of Quality of Experience , 2013 .

[21]  Alexander Raake,et al.  Quality of Experience , 2014, T-Labs Series in Telecommunication Services.

[22]  Fabrizio Cutolo,et al.  The Role of Camera Convergence in Stereoscopic Video See-through Augmented Reality Displays , 2018 .

[23]  Steven K. Feiner,et al.  Perceptual issues in augmented reality revisited , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[24]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[25]  Ruzena Bajcsy,et al.  User experience and interaction performance in 2D/3D telecollaboration , 2018, Future Gener. Comput. Syst..

[26]  Hooman Hedayati,et al.  Robot Teleoperation with Augmented Reality Virtual Surrogates , 2019, 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[27]  Sean Brennan,et al.  Comparing User QoE via Physiological and Interaction Measurements of Immersive AR and VR Speech and Language Therapy Applications , 2017, ACM Multimedia.

[28]  Naokazu Yokoya,et al.  Augmented telepresence using autopilot airship and omni-directional camera , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[29]  Michael Beigl,et al.  A study of depth perception in hand-held augmented reality using autostereoscopic displays , 2014, 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[30]  Wen Yi,et al.  A critical review of virtual and augmented reality (VR/AR) applications in construction safety , 2018 .

[31]  Peter Kazanzides,et al.  Scene Modeling and Augmented Virtuality Interface for Telerobotic Satellite Servicing , 2018, IEEE Robotics and Automation Letters.

[32]  Marc Hassenzahl,et al.  User experience (UX): towards an experiential perspective on product quality , 2008, IHM '08.

[33]  Ashutosh Singla,et al.  Subjective quality evaluation of tile-based streaming for omnidirectional videos , 2019, MMSys.

[34]  Chiuhsiang Joe Lin,et al.  Interaction and visual performance in stereoscopic displays: A review , 2015 .

[35]  Raimund Schatz,et al.  Assessing the QoE Impact of 3D Rendering Style in the Context of VR-based Training , 2018, 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX).

[36]  ITU-T Rec. P.10/G.100 (11/2017) Vocabulary for performance, quality of service and quality of experience , 2017 .

[37]  M. Tscheligi,et al.  Questionnaires embedded in virtual environments: reliability and positioning of rating scales in virtual environments , 2019, Quality and User Experience.

[38]  Nassir Navab,et al.  Towards Efficient Visual Guidance in Limited Field-of-View Head-Mounted Displays , 2018, IEEE Transactions on Visualization and Computer Graphics.

[39]  Touradj Ebrahimi,et al.  Towards an efficient methodology for evaluation of quality of experience in Augmented Reality , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[40]  Danielle Albers Szafir,et al.  Designing for Depth Perceptions in Augmented Reality , 2017, 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[41]  H. V. D. Loos,et al.  Orthographic Vision-based Interface for Robot Arm Teleoperation , 2018 .

[42]  Tiago M. Fernández-Caramés,et al.  A Review on Industrial Augmented Reality Systems for the Industry 4.0 Shipyard , 2018, IEEE Access.

[43]  Mathias Wien,et al.  Standardization Status of Immersive Video Coding , 2019, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[44]  D. Altman,et al.  Multiple significance tests: the Bonferroni method , 1995, BMJ.

[45]  Teruhisa Misu,et al.  Augmented Reality Interface Design Approaches for Goal-directed and Stimulus-driven Driving Tasks , 2018, IEEE Transactions on Visualization and Computer Graphics.

[46]  Andrea Albarelli,et al.  On the Interplay between Data Overlay and Real-World Context using See-through Displays , 2015, CHItaly.

[47]  Nigel Bevan,et al.  Classifying and selecting UX and usability measures , 2008 .

[48]  Miguel Barreda-Ángeles,et al.  Users’ psychophysiological, vocal, and self-reported responses to the apparent attitude of a virtual audience in stereoscopic 360°-video , 2019, Virtual Reality.

[49]  Yuan Li,et al.  Evaluation of Environment-Independent Techniques for 3D Position Marking in Augmented Reality , 2018, 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[50]  Truong Cong Thang,et al.  A Subjective Study on User Perception Aspects in Virtual Reality , 2019 .

[51]  Ashutosh Singla,et al.  Measuring and comparing QoE and simulator sickness of omnidirectional videos in different head mounted displays , 2017, 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX).

[52]  Takanori Hayashi,et al.  Performance comparison of subjective assessment methods for 3D video quality , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[53]  Cheng-Hsin Hsu,et al.  A Survey on 360° Video Streaming , 2019, ACM Comput. Surv..

[54]  David Harborth,et al.  Augmented Reality - a Game Changing Technology for manufacturing Processes? , 2018, ECIS.

[55]  Igor Danilo Diego Curcio,et al.  360-Degree Video Streaming and Its Subjective Quality , 2017 .

[56]  ITU-T Rec. P.910 (04/2008) Subjective video quality assessment methods for multimedia applications , 2009 .

[57]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[58]  Yongtian Wang,et al.  Subjective and objective evaluation of visual fatigue caused by continuous and discontinuous use of HMDs , 2018, Journal of the Society for Information Display.

[59]  Sebastian Möller,et al.  Formal Definition of QoE Metrics , 2016, ArXiv.

[60]  Touradj Ebrahimi,et al.  Modeling immersive media experiences by sensing impact on subjects , 2015, Multimedia Tools and Applications.

[61]  Sebastian Bosse,et al.  A common framework for the evaluation of psychophysiological visual quality assessment , 2019 .

[62]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[63]  Jacek Malec,et al.  Remote Supervision of an Unmanned Surface Vessel - A Comparison of Interfaces , 2019, 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[64]  Ross T. Smith,et al.  Cognitive Cost of Using Augmented Reality Displays , 2017, IEEE Transactions on Visualization and Computer Graphics.

[65]  Teruhisa Misu,et al.  Driver Behavior and Performance with Augmented Reality Pedestrian Collision Warning: An Outdoor User Study , 2018, IEEE Transactions on Visualization and Computer Graphics.

[66]  Maja Matijasevic,et al.  Performance and QoE assessment of HTC Vive and Oculus Rift for pick-and-place tasks in VR , 2017, 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX).

[67]  Niall Murray,et al.  A quality of experience evaluation system and research challenges for networked virtual reality-based teleoperation applications , 2019, MMVE@MMSys.

[68]  Carlo Alberto Avizzano,et al.  Effects of Augmented Reality on the Performance of Teleoperated Industrial Assembly Tasks in a Robotic Embodiment , 2018, IEEE Transactions on Human-Machine Systems.

[69]  Carlo Alberto Avizzano,et al.  A Stereo-Panoramic Telepresence System for Construction Machines☆ , 2017 .

[70]  Muhammad Imran,et al.  Quality of Experience for a Virtual Reality simulator , 2018, HVEI.

[71]  Hirokazu Kato,et al.  Augmented Reality versus Virtual Reality for 3D Object Manipulation , 2018, IEEE Transactions on Visualization and Computer Graphics.

[72]  Kjell Brunnström,et al.  Subjective evaluation of a 3D videoconferencing system , 2014, Electronic Imaging.

[73]  Doug A. Bowman,et al.  Effect of Volumetric Displays on Depth Perception in Augmented Reality , 2018, AutomotiveUI.

[74]  Marcus Barkowsky,et al.  Statistical quality of experience analysis: on planning the sample size and statistical significance testing , 2018, J. Electronic Imaging.

[75]  Mattias Andersson,et al.  View Position Impact on QoE in an Immersive Telepresence System for Remote Operation , 2019, 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX).

[76]  Sebastian Bosse,et al.  Psychophysiology-Based QoE Assessment: A Survey , 2017, IEEE Journal of Selected Topics in Signal Processing.

[77]  Pirkko Oittinen,et al.  Stereoscopic depth perception in video see-through augmented reality within action space , 2014, J. Electronic Imaging.

[78]  Silvio Ricardo Rodrigues Sanches,et al.  Influence of the Camera Viewpoint on Augmented Reality Interaction , 2018, IEEE Latin America Transactions.

[79]  Ronald Marsh,et al.  Survey on depth perception in head mounted displays: distance estimation in virtual reality, augmented reality, and mixed reality , 2019, IET Image Process..

[80]  Federico Guedea-Elizalde,et al.  Marker’s position estimation under uncontrolled environment for augmented reality , 2016, International Journal on Interactive Design and Manufacturing (IJIDeM).

[81]  Mattias Andersson,et al.  Quality of Experience of hand controller latency in a virtual reality simulator , 2019, HVEI.