Vision Interfaces and Systems Laboratory Technical Report VISLab-94-001 October , 1994 FingerMouse : A Freehand Computer Pointing Interface

FingerMouse is a freehand pointing alternative to the ubiquitous mouse. In this system, the user merely performs a pointing gesture above the keyboard. A down-looking camera is trained on the keyboard. A user typing at the keyboard may switch into the 'pointing mode' by simply assuming a hand pointing configuration above the keyboard. A vision system constantly monitors the hand and tracks the fingertip of the pointing hand. As the user gestures in a horizontal plane just above the keyboard, the screen cursor moves accordingly. The user depresses the SHIFT key on the keyboard with the non-pointing hand to register a 'mouse button press'. This report presents a probabilistic color-based segmentation algorithm which transforms a color transfer function into probability entries in a RGB lookup table. It details a finite-state machine approach for the rapid detection of the pointing hand and describes a principal axis-based approach for fingertip tracking. The system has achieved near real-time performance on a general purpose workstation. Preliminary user testing results are presented which suggest that FingerMouse is a promising candidate as a general purpose mouse-type input device.