Extending Upper Limb User Interactions in AR, VR and MR Headsets Employing a Custom-Made Wearable Device

Upper limb interactions play an important role in virtual, augmented, and mixed reality scenarios. Numerous sensors including optical, magnetic, mechanical, and myography, have been employed to provide more natural interactions in comparison to game controllers. Recently, virtual, augmented, and mixed reality headsets have started embedding hand tracking sensors to simplify the hardware requirements, thus easing operation and setup. The tracking integration is being referred to as inside/out tracking, whereby hand and eye tracking interactions without external sensors or controllers provide interactive freedom. However, the motion capture area of the inside/out sensors is limited to the field of view and technical features of the cameras, and restricted to a few hand tracking gestures. In this paper, we introduce a custom upper limb motion tracking device that extends the user's interaction range while employing a virtual, augmented, or mixed reality headset. Our 3D motion tracking system is a compact wireless wearable prototype that uses inertial measurement units providing orientation and position data employed for upper limb user interaction outside the field of view of the inside/out sensors.