Construction of Skin Color Map for Resolving Hand Occlusion in AR Environments

Received 8 January 2014; received in revised form 26 March 2014; accepted 8 April 2014ABSTRACTIn tangible augmented reality (AR) environments, the user interacts with virtual objects bymanipulating their physical counterparts, but he or she often encounters awkward situations inwhich his or her hands are occluded by the augmented virtual objects, which causes great diffi-culty in figuring out hand positions, and reduces both immersion and ease of interaction. Tosolve the problem of such hand occlusion, skin color information has been usefully exploited. Inthis paper, we propose an approach to simple and effective construction of a skin color mapwhich is suitable for hand segmentation and tangible AR interaction. The basic idea used hereinis to obtain hand images used in a target AR environment by simple image subtraction and torepresent their color information by a convex polygonal map in the YCbCr color space. Weexperimentally found that the convex polygonal map is more accurate in representing skin colorthan a conventional rectangular map. After implementing a solution for resolving hand occlu-sion using the proposed skin color map construction, we showed its usefulness by applying it tovirtual design evaluation of digital handheld products in a tangible AR environment.Key Words:Augmented reality interaction, Convex polygon, Hand occlusion, Image processing,Skin color map, YCbCr color space