Real-Time Robotic Hand Control Using Hand Gestures

this paper presents a new approach for controlling robotic hand or an individual robot by merely showing hand gestures in front of a camera. With the help of this technique one can pose a hand gesture in the vision range of a robot and corresponding to this notation, desired action is performed by the robotic system. Simple video camera is used for computer vision, which helps in monitoring gesture presentation. This approach consists of four modules: (a) A real time hand gesture formation monitor and gesture capture, (b) feature extraction, (c) Pattern matching for gesture recognition, (d) Command determination corresponding to shown gesture and performing action by robotic system. Real-time hand tracking technique is used for object detection in the range of vision. If a hand gesture is shown for one second, the camera captures the gesture. Object of interest is extracted from the background and the portion of hand, representing the gesture, is cropped out using the statistical property of hand. Extracted hand gesture is matched with the stored database of hand gestures using pattern matching. Corresponding to the matched gesture, action is performed by the robot

[1]  David Kortenkamp,et al.  Recognizing and Interpreting Gestures on a Mobile Robot , 1996, AAAI/IAAI, Vol. 2.

[2]  Tiehan Lu,et al.  Design of a real-time gesture recognition system: high performance through algorithms and software , 2005, IEEE Signal Processing Magazine.

[3]  Daggu Venkateshwar Rao,et al.  Implementation and Evaluation of Image Processing Algorithms on Reconfigurable Architecture using C-based Hardware Descriptive Languages , .

[4]  Nalini K. Ratha,et al.  FPGA-based computing in computer vision , 1997, Proceedings Fourth IEEE International Workshop on Computer Architecture for Machine Perception. CAMP'97.

[5]  Jamshid Dastur,et al.  Robotic Arm Actuation with 7 DOF Using Haar Classifier Gesture Recognition , 2010, 2010 Second International Conference on Computer Engineering and Applications.

[6]  Dmitry B. Goldgof,et al.  Gesture recognition using Bezier curves for visualization navigation from registered 3-D data , 2004, Pattern Recognit..

[7]  Helge J. Ritter,et al.  Parametrized SOMs for hand posture reconstruction , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[8]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[9]  William T. Freeman.  Computer vision for television and games , 1999, Proceedings International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems. In Conjunction with ICCV'99 (Cat. No.PR00378).

[10]  Stephan Wong,et al.  Vision­Based Hand Gesture Recognition for Human Computer Interaction: A Review , 2008 .

[11]  Ying Wu,et al.  Vision-Based Gesture Recognition: A Review , 1999, Gesture Workshop.

[12]  Yoichi Sato,et al.  Real-Time Fingertip Tracking and Gesture Recognition , 2002, IEEE Computer Graphics and Applications.

[13]  Sean Gallagher,et al.  CASE-STUDY OF A XILINX SYSTEM GENERATOR DESIGN FLOW FOR RAPID DEVELOPMENT OF SDR WAVEFORMS , 2005 .

[14]  Surendra Ranganath,et al.  Real-time gesture recognition system and application , 2002, Image Vis. Comput..

[15]  Aditya Ramamoorthy,et al.  Recognition of dynamic hand gestures , 2003, Pattern Recognit..

[16]  Rohit Verma,et al.  Vision based hand gesture recognition using finite state machines and fuzzy logic , 2009, 2009 International Conference on Ultra Modern Telecommunications & Workshops.

[17]  Paulo Menezes,et al.  Face tracking and hand gesture recognition for human-robot interaction , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[18]  Jayantha Katupitiya,et al.  Implementation of a PC based controller for a PUMA robot , 1997, Proceedings Fourth Annual Conference on Mechatronics and Machine Vision in Practice.

[19]  Mubarak Shah,et al.  Recognizing Hand Gestures , 1994, ECCV.

[20]  Klaus Hartmann,et al.  Real Time Hand Based Robot Control Using Multimodal Images , 2008 .

[21]  Franklin C. Crow,et al.  Summed-area tables for texture mapping , 1984, SIGGRAPH.

[22]  Chris Joslin,et al.  A Framework for 3D Hand Tracking and Gesture Recognition using Elements of Genetic Programming , 2007, Fourth Canadian Conference on Computer and Robot Vision (CRV '07).

[23]  Toyoaki Nishida,et al.  Unsupervised simultaneous learning of gestures, actions and their associations for Human-Robot Interaction , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  Helge J. Ritter,et al.  Visual recognition of continuous hand postures , 2002, IEEE Trans. Neural Networks.

[25]  Seung-Hong Hong,et al.  Hand Region Extraction and Gesture Recognition using entropy analysis , 2006 .

[26]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Bogdan Ionescu,et al.  Gestures Recognition Based on the Fusion of Hand Positioning and Arm Gestures , 2006, J. Robotics Mechatronics.

[28]  Fang Zhong,et al.  Parallel architecture for PCA image feature detection using FPGA , 2008, 2008 Canadian Conference on Electrical and Computer Engineering.

[29]  C CrowFranklin.  Summed-area tables for texture mapping , 1984 .

[30]  V.M. Becerra,et al.  Hardware retrofit and computed torque control of a Puma 560 Robot updating an industrial manipulator , 2004, IEEE Control Systems.

[31]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[32]  Konstantinos G. P. Derpanis,et al.  A Review of Vision-Based Hand Gestures , 2004 .

[33]  Jochen Triesch,et al.  Robotic Gesture Recognition , 1997, Gesture Workshop.

[34]  R. Ishii,et al.  Hand shape extraction from a sequence of digitized gray-scale images , 1994, Proceedings of IECON'94 - 20th Annual Conference of IEEE Industrial Electronics.