For Vision-Based Human Computer Interaction

In the evolution of user interfaces, keyboards were the primary devices in text-based user interfaces, and then the invention of the mouse brought us the graphical user interface. What is the counterpart of the mouse when we are trying to explore three-dimensional (3-D) virtual environments (VEs)? In many current VE applications, keyboards, mice, wands, and joysticks are the common controlling and navigating devices. However, to some extent, such mechanical devices are inconvenient and unsuitable for natural and direct interaction, because it is difficult for these devices to supply 3-D and high degree of freedom inputs. Although magnetic trackers are being used as sensors for 3-D inputs in some of these devices, they are prone to magnetic interference, and they only supply global motion information. A more convenient and natural device is desirable to achieve more immersive interaction. The use of hand gestures has become an important part of human computer interaction (HCI) in recent years [1], [24]. To use human hands as a natural interface device, some glove-based devices have been employed to capture human hand motion by attaching sensors to measure the joint angles and spatial positions of hands directly. Unfortunately, such devices are expensive and cumbersome. Since rich visual information provides a strong cue to infer the inner states of an object, vision-based techniques provide promising alternatives to capture human hand motion. At the same time, vision systems could be very cost efficient and noninvasive. These facts serve as the motivating forces for research in the modeling, analysis, animation, and recognition of hand gestures. According to different application scenarios, hand gestures can be classified into several categories: conversational gestures, controlling gestures, manipulative gestures and communicative gestures. Sign language is an important case of communicative gestures. Because sign languages are highly structured [33], [37], they are very suitable as a test-bed for vision algorithms [33], [37]. Controlling gestures are the focus of current research in vision-based interfaces [6], [17], [23], [26], [35], [45]. Virtual objects can be located by analyzing pointing gestures [24]. Some display-control applications demon-