Effects of device obtrusion and tool-hand misalignment on user performance and stiffness perception in visuo-haptic mixed reality

Abstract The Visuo-Haptic Mixed Reality (VHMR) is a branch of the Mixed Reality (MR) that is acquiring more and more interest in the recent years. Its success is due to the ability of merging visual and tactile perceptions of both virtual and real objects with a collocated approach. Like any emerging technology, the development of the VHMR systems is accompanied by challenges that, in this case, deals with the efforts to enhance the multi-modal human perception with the user-computer interface and interaction devices at the moment available. This paper deals with two of the typical problems related to VHMR systems, that are device obtrusion and tool–hand misalignment, and suggests solutions whose effectiveness has been tested by means of user studies. First, the paper analyzes the obtrusion problem and the benefits that users may gain performing task in a mixed environment with unobstructed haptic feedback, performed by means of a novel technique. Secondly, it investigates the effects of tool–hand misalignment on user perception and verifies the efficacy of a proposed misalignment correction technique by means of a comparative user test. Experimental results show that users would benefit from using the proposed unobstructed visuo-haptic approach and the misalignment compensation technique. These enhancements demonstrate the efficacy of the proposed solutions and at the same time get stronger the awareness that obtrusion and misalignment problems are fundamental issues to take into account for producing a realistic perception of a visuo-haptic mixed environment.