Developing a smart camera for gesture recognition in HCI applications

Smart cameras are becoming more popular in human computer interaction (HCI). One of HCI research areas, multimodal user interface (MMUI) allows user to interact with a computer by using his or her natural communication modalities, such as speech, pen, touch, gestures, eye gaze, and facial expression. This paper presents the hardware and software co-design and implementation of KLT (Kanade Lucas Tomasi) tracking algorithm in a FPGA-based smart camera prototype for recognize simple hand gestures consisting of a CMOS image sensor capture unit and FPGA main video processor. This tracking system uses face and hand detections as a tool to detect and track gesture (face and hand motion). This gesture tracking system that are based on Harris Keypoint Detection algorithm and KLT tracking algorithm has been successfully implemented in FPGA and shows promising result in real time performance.

[1]  Nuria Oliver,et al.  GWindows: robust stereo vision for gesture-based control of windows , 2003, ICMI '03.

[2]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[3]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[4]  Yu Shi,et al.  Developing a smart camera for road traffic surveillance , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[5]  Laurent Letellier,et al.  Smart camera design for intensive embedded computing , 2005, Real Time Imaging.

[6]  Wayne H. Wolf,et al.  Smart Cameras as Embedded Systems , 2002, Computer.

[7]  Bernhard Rinner,et al.  An Embedded Smart Camera on a Scalable Heterogeneous Multi-DSP System , 2004 .

[8]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[9]  C Tomasi,et al.  Shape and motion from image streams: a factorization method. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[10]  João M. P. Cardoso,et al.  A Real Time Gesture Recognition System for Mobile Robots , 2004, ICINCO.