XRgonomics: Facilitating the Creation of Ergonomic 3D Interfaces

Arm discomfort is a common issue in Cross Reality applications involving prolonged mid-air interaction. Solving this problem is difficult because of the lack of tools and guidelines for 3D user interface design. Therefore, we propose a method to make existing ergonomic metrics available to creators during design by estimating the interaction cost at each reachable position in the user’s environment. We present XRgonomics, a toolkit to visualize the interaction cost and make it available at runtime, allowing creators to identify UI positions that optimize users’ comfort. Two scenarios show how the toolkit can support 3D UI design and dynamic adaptation of UIs based on spatial constraints. We present results from a walkthrough demonstration, which highlight the potential of XRgonomics to make ergonomics metrics accessible during the design and development of 3D UIs. Finally, we discuss how the toolkit may address design goals beyond ergonomics.

[1]  Antti Oulasvirta,et al.  Is motion capture-based biomechanical simulation valid for HCI studies?: study and implications , 2014, CHI.

[2]  Eyal Ofek,et al.  SnapToReality: Aligning Augmented Reality to the Real World , 2016, CHI.

[3]  Ayman Habib,et al.  OpenSim: Open-Source Software to Create and Analyze Dynamic Simulations of Movement , 2007, IEEE Transactions on Biomedical Engineering.

[4]  Dieter Schmalstieg,et al.  Adaptive information density for augmented reality displays , 2016, 2016 IEEE Virtual Reality (VR).

[5]  Hans-Jörg Schulz,et al.  Beyond Trust Building — Calibrating Trust in Visual Analytics , 2020, 2020 IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX).

[6]  Krzysztof Z. Gajos,et al.  Automatically generating personalized user interfaces with Supple , 2010, Artif. Intell..

[7]  David Lindlbauer,et al.  Context-Aware Online Adaptation of Mixed Reality Interfaces , 2019, UIST.

[8]  Saul Greenberg,et al.  Evaluation Strategies for HCI Toolkit Research , 2018, CHI.

[9]  Antti Oulasvirta,et al.  Informing the Design of Novel Input Methods with Muscle Coactivation Clustering , 2015, ACM Trans. Comput. Hum. Interact..

[10]  Marko Jurmu,et al.  Scroll, tilt or move it: using mobile phones to continuously control pointers on large public displays , 2009, OZCHI.

[11]  Gisela Sjøgaard,et al.  Muscle blood flow during isometric activity and its relation to muscle fatigue , 2004, European Journal of Applied Physiology and Occupational Physiology.

[12]  L McAtamney,et al.  RULA: a survey method for the investigation of work-related upper limb disorders. , 1993, Applied ergonomics.

[13]  Jörg Müller,et al.  Ownershift: Facilitating Overhead Interaction in Virtual Reality with an Ownership-Preserving Hand Space Shift , 2018, UIST.

[14]  Ajay Seth,et al.  Is my model good enough? Best practices for verification and validation of musculoskeletal models and simulations of movement. , 2015, Journal of biomechanical engineering.

[15]  Claire C. Gordon,et al.  2012 Anthropometric Survey of U.S. Army Personnel: Methods and Summary Statistics , 2014 .

[16]  Pushmeet Kohli,et al.  FLARE: Fast layout for augmented reality applications , 2014, 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[17]  Anca Velisar,et al.  Benchmarking of dynamic simulation predictions in two software platforms using an upper limb musculoskeletal model , 2015, Computer methods in biomechanics and biomedical engineering.

[18]  Andrés Lucero,et al.  May AI?: Design Ideation with Cooperative Contextual Bandits , 2019, CHI.

[19]  G. Borg Psychophysical scaling with applications in physical work and the perception of exertion. , 1990, Scandinavian journal of work, environment & health.

[20]  Holger Regenbrecht,et al.  Towards Pervasive Augmented Reality: Context-Awareness in Augmented Reality , 2017, IEEE Transactions on Visualization and Computer Graphics.

[21]  Gerd Hirzinger,et al.  Capturing robot workspace structure: representing robot capabilities , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[22]  Pourang Irani,et al.  Consumed endurance: a metric to quantify arm fatigue of mid-air interactions , 2014, CHI.

[23]  Stephen DiVerdi,et al.  Level of detail interfaces , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[24]  Hans-Peter Kriegel,et al.  The R*-tree: an efficient and robust access method for points and rectangles , 1990, SIGMOD '90.

[25]  Karthik Ramani,et al.  Modeling Cumulative Arm Fatigue in Mid-Air Interaction based on Perceived Exertion and Kinetics of Arm Motion , 2017, CHI.

[26]  Mario Cifrek,et al.  Surface EMG based muscle fatigue evaluation in biomechanics. , 2009, Clinical biomechanics.

[27]  James Carifio,et al.  Ten Common Misunderstandings, Misconceptions, Persistent Myths and Urban Legends about Likert Scales and Likert Response Formats and their Antidotes , 2007 .

[28]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[29]  Sriram Subramanian,et al.  Erg-O: Ergonomic Optimization of Immersive Virtual Environments , 2017, UIST.

[30]  Antti Oulasvirta,et al.  Combinatorial Optimization of Graphical User Interface Designs , 2020, Proceedings of the IEEE.

[31]  Antti Oulasvirta,et al.  Sketchplore: Sketch and Explore with a Layout Optimiser , 2016, Conference on Designing Interactive Systems.

[32]  Antti Oulasvirta,et al.  MenuOptimizer: interactive optimization of menu systems , 2013, UIST.

[33]  Samuel B. Williams,et al.  Association for Computing Machinery , 2009 .

[34]  Joachim Deisinger,et al.  ERGONAUT: a tool for ergonomic analyses in virtual environments , 2000 .

[35]  John T. McConville,et al.  Anthropometric Survey of U.S. Army Personnel: Methods and Summary Statistics 1988 , 1989 .

[36]  Marc Alexa,et al.  OptiSpace: Automated Placement of Interactive 3D Projection Mapping Content , 2018, CHI.

[37]  David A. Rosenbaum Chapter 2 – Core Problems , 2010 .

[38]  Ivan Poupyrev,et al.  3D User Interfaces: Theory and Practice , 2004 .

[39]  Andrea Bunt,et al.  Creating Augmented and Virtual Reality Applications: Current Practices, Challenges, and Opportunities , 2020, CHI.

[40]  Brad A. Myers,et al.  Past, Present and Future of User Interface Software Tools , 2000, TCHI.

[41]  Daniel Vogel,et al.  Gunslinger: Subtle Arms-down Mid-air Interaction , 2015, UIST.

[42]  J. Saunders,et al.  Observations of the Function of the Shoulder Joint , 1996, Clinical orthopaedics and related research.

[43]  Norman I. Badler,et al.  Real-Time Inverse Kinematics of the Human Arm , 1996, Presence: Teleoperators & Virtual Environments.