3D Hand Gesture Recognition Based on Sensor Fusion of Commodity Hardware

With the advent of various video game consoles and tablet devices gesture recognition got quite popular to control computer systems. E.g. touch screens allow for an intuitive control of small 2d user interfaces with finger gestures. For interactive manipulation of 3d objects in a large 3d projection environment a similar intuitive 3d interaction method is required. In this paper, we present a dynamic 3d hand and arm gesture recognition system using commodity hardware. The input data is captured with low cost depth sensors (e.g. Microsoft Kinect) and HD color sensors (e.g. Logitech C910). Our method combines dynamic hand and arm gesture recognition based on the depth sensor with static hand gesture recognition based on the HD color sensor.

[1]  Sebastian Thrun,et al.  Real time motion capture using a single time-of-flight camera , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Esther M. Arkin,et al.  An efficiently computable metric for comparing polygonal shapes , 1991, SODA '90.

[3]  S. E. Ghobadi,et al.  Hand Segmentation Using 2D/3D Images , 2007 .

[4]  Nong Sang,et al.  Real-time skin color detection under rapidly changing illumination conditions , 2011, IEEE Transactions on Consumer Electronics.

[5]  D. Wormell,et al.  Advancements in 3D interactive devices for virtual environments , 2003 .

[6]  Mario Hernández-Tejera,et al.  Hand Gesture Recognition for Human-Machine Interaction , 2004, WSCG.

[7]  Klaus Hartmann,et al.  Real Time Hand Based Robot Control Using Multimodal Images , 2008 .

[8]  David H. Douglas,et al.  ALGORITHMS FOR THE REDUCTION OF THE NUMBER OF POINTS REQUIRED TO REPRESENT A DIGITIZED LINE OR ITS CARICATURE , 1973 .

[9]  Michael G. Strintzis,et al.  A gesture recognition system using 3D data , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[10]  Chee Seng Chan,et al.  A Fusion Approach for Efficient Human Skin Detection , 2012, IEEE Transactions on Industrial Informatics.

[11]  Sebastian Thrun,et al.  Real-time identification and localization of body parts from depth images , 2010, 2010 IEEE International Conference on Robotics and Automation.

[12]  Chun-Jen Chen,et al.  A component-labeling algorithm using contour tracing technique , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[13]  David Zeltzer,et al.  A survey of glove-based input , 1994, IEEE Computer Graphics and Applications.

[14]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[15]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.