A Close-Range Gesture Interaction with Kinect

A large number of tracking and gesture recognition algorithms and technologies have been developed in the field of human-computer interactions thanks to the introduction of cameras with depth sensors such as Microsoft's Kinect. Most of the techniques rely on skeleton tracking which is more suitable for distant and full body interaction. This paper presents a new real-time finger-gesture interaction system using Kinect v2 that identifies fingertips and finger gestures that enable the natural user interaction at a close distance. Our contribution also includes various gesture recognition algorithms using two and three fingers such as L-gesture, OK-gesture, Rock-gesture and Scissor-gesture, in addition to full hand and one-finger gestures. We demonstrate the effectiveness of our system through a fruit slicing game and compare the results to the Leap Motion device.

[1]  Gokula Krishna.  Finger Tracking In Real Time Human Computer Interaction , 2014 .

[2]  Ankit Chaudhary,et al.  Tracking of Fingertips and Centers of Palm Using KINECT , 2011, 2011 Third International Conference on Computational Intelligence, Modelling & Simulation.

[3]  Baining Guo,et al.  Exemplar-based human action pose correction and tagging , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Gregory Goth.  Brave NUI world , 2011, CACM.

[5]  Yu Youling,et al.  Finger Tracking and Gesture Recognition with Kinect , 2012, 2012 IEEE 12th International Conference on Computer and Information Technology.

[6]  Bart Selman,et al.  Unstructured human activity detection from RGBD images , 2011, 2012 IEEE International Conference on Robotics and Automation.

[7]  Junsong Yuan,et al.  Robust hand gesture recognition with kinect sensor , 2011, ACM Multimedia.

[8]  Paolo Dario,et al.  A Survey of Glove-Based Systems and Their Applications , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[9]  Andrew W. Fitzgibbon,et al.  Efficient regression of general-activity human poses from depth images , 2011, 2011 International Conference on Computer Vision.

[10]  Daniel Thalmann,et al.  3D fingertip and palm tracking in depth image sequences , 2012, ACM Multimedia.

[11]  Dennis Wixon,et al.  The Natural User Interface , 2011 .

[12]  Hui-mei Justina Hsu.  The Potential of Kinect in Education , 2011 .

[13]  Albert A. Rizzo,et al.  Interactive game-based rehabilitation using the Microsoft Kinect , 2012, 2012 IEEE Virtual Reality Workshops (VRW).

[14]  Ankit Chaudhary,et al.  A Vision based Geometrical Method to find Fingers Positions in Real Time Hand Gesture Recognition , 2012, J. Softw..

[15]  R. Cowan,et al.  Showing or telling? Local interaction and organization of behavior , 2013, Journal of Economic Interaction and Coordination.

[16]  Maneesh Agrawala,et al.  3D puppetry: a kinect-based interface for 3D animation , 2012, UIST.

[17]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[18]  Murray Griffin,et al.  Activity-promoting gaming systems in exercise and rehabilitation. , 2011, Journal of rehabilitation research and development.

[19]  Kurt Jaegers.  XNA 4.0 game development by example : beginner's guide : create exciting games with Microsoft XNA 4.0 , 2010 .

[20]  Toyin Osunkoya,et al.  Gesture-Based Human-Computer-Interaction Using Kinect for Windows Mouse Control and PowerPoint Presentation , 2013 .

[21]  Ling Shao,et al.  Enhanced Computer Vision With Microsoft Kinect Sensor: A Review , 2013, IEEE Transactions on Cybernetics.

[22]  Jiro Tanaka,et al.  Finger identification and hand gesture recognition techniques for natural user interface , 2013, APCHI.

[23]  Stan Sclaroff,et al.  A Unified Framework for Gesture Recognition and Spatiotemporal Gesture Segmentation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Matthew Tang,et al.  Recognizing Hand Gestures with Microsoft ’ s Kinect , 2011 .

[25]  Toby Sharp,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR.

[26]  Tae-Seong Kim,et al.  Human Activity Recognition via the Features of Labeled Depth Body Parts , 2012, ICOST.

[27]  Massimo Panella,et al.  An Accurate Algorithm for the Identification of Fingertips Using an RGB-D Camera , 2013, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[28]  K. Hawick,et al.  Comparing Hand-Gesture and Finger-Touch Interfacing to Navigate Bulk Image-Sequence Data , 2013 .

[29]  Kun Qian,et al.  Developing a Gesture Based Remote Human-Robot Interaction System Using Kinect , 2013 .

[30]  K. Kinage.  Study on Hand Gesture Recognition , 2015 .

[31]  Mario Ciampi,et al.  Controller-free exploration of medical image data: Experiencing the Kinect , 2011, 2011 24th International Symposium on Computer-Based Medical Systems (CBMS).

[32]  Mario Fernando Montenegro Campos,et al.  Real-Time Gesture Recognition from Depth Data through Key Poses Learning and Decision Forests , 2012, 2012 25th SIBGRAPI Conference on Graphics, Patterns and Images.

[33]  Pan Jing,et al.  Human-computer Interaction using Pointing Gesture based on an Adaptive Virtual Touch Screen , 2013 .

[34]  Rafael Mira De Oliveira Libardi,et al.  Design and evaluation case study: evaluating the kinect device in the task of natural interaction in a visualization system , 2014 .

[35]  Yoichi Sato,et al.  Real-Time Fingertip Tracking and Gesture Recognition , 2002, IEEE Computer Graphics and Applications.

[36]  Stavros J. Perantonis,et al.  Hand Shape and 3D Pose Estimation Using Depth Data from a Single Cluttered Frame , 2012, ISVC.

[37]  Rod McCall,et al.  Lightweight palm and finger tracking for real-time 3D gesture control , 2011, 2011 IEEE Virtual Reality Conference.

[38]  Yi Li,et al.  Hand gesture recognition using Kinect , 2012, 2012 IEEE International Conference on Computer Science and Automation Engineering.

[39]  Joseph J. LaViola,et al.  Poster: Real-time markerless kinect based finger tracking and hand gesture recognition for HCI , 2013, 2013 IEEE Symposium on 3D User Interfaces (3DUI).

[40]  Sai-Keung Wong,et al.  Interactive Sand Art Drawing Using Kinect , 2014, VINCI '14.