Capturing human hand movements with a webcam to control an anthropomorphic gripper

Abstract Currently all method types used to control an anthropomorphic gripper can be grouped into two classes, namely: methods based on motion capture using various capturing devices (e.g.: data gloves-based method types) and methods based on motion capture using different algorithms applied to digital images. Approaches based on the utilization of data gloves can digitize human hands and fingers motion in input parameters for a virtual reality system. Approaches that use image processing algorithms utilize on the other hand image capture devices or depth data capture devices to digitize the human hand and its movements. Using the second method the communications between man and computer becomes much more natural and given the recent advancements in this field, is tending to become the normal form of interaction. This paper presents the components of a system used to integrate the human hand movement into a virtual reality environment using a method based on a boosting algorithm. Boosting algorithms are used, in the proposed system described in this paper, for the detection of human hand and its difference gestures. The device used to capture images is a Web camera that captures the data and feeds it into the system for further processing. The processing result is used to control a virtual anthropomorphic gripper that will duplicate several gestures in virtual environment. This paper presents the main hardware and software components that were obtained, the system implementation, and it’s testing, evaluate robustness and the ability to accurately spot the human hand.