Robot arm teleoperation via RGBD sensor palm tracking

In this paper we present a simple and intuitive approach to teleoperating a 3 or higher degrees-of-freedom (DOF) robotic arm in Cartesian space. Using an RGBD camera, we retrieve the position of the user's palm. This position is then translated into the desired robotic arm position, which is then used as an input to a control loop. The entire system is implemented in the Robotic Operating System, enabling simple functionality transfer to any compatible robotic arm. The system was tested on the Kinova Jaco 6DOF robotic arm with the aim of using it for object manipulation. We use the inverse kinematics for calculating the joint rotation velocities required for following the Cartesian path of the human hand. The resulting joint velocities are then sent to the robotic arm control interface which then passes commands to the pertaining API. Results corroborate the validity of the proposed approach for robotic arm teleoperation, opening the possibility for many potential applications.

[1]  Takeo Kanade,et al.  Visual Tracking of High DOF Articulated Structures: an Application to Human Hand Tracking , 1994, ECCV.

[2]  Paulo R. S. Mendonça,et al.  Model-based 3D tracking of an articulated hand , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[3]  David J. Fleet,et al.  Model-Based 3D Hand Pose Estimation from Monocular Video , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[5]  Gourab Sen Gupta,et al.  Master–Slave Control of a Teleoperated Anthropomorphic Robotic Arm With Gripping Force Sensing , 2006, IEEE Transactions on Instrumentation and Measurement.

[6]  Jürgen Schmidhuber,et al.  Teleoperation of a 7 DOF humanoid robot arm using human arm accelerations and EMG signals , 2014 .

[7]  Panagiotis K. Artemiadis,et al.  EMG-Based Control of a Robot Arm Using Low-Dimensional Embeddings , 2010, IEEE Transactions on Robotics.

[8]  Benjamin C. Kuo,et al.  AUTOMATIC CONTROL SYSTEMS , 1962, Universum:Technical sciences.

[9]  Frank Chongwoo Park,et al.  Movement Primitives, Principal Component Analysis, and the Efficient Generation of Natural Motions , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[10]  Ken Perlin,et al.  Real-Time Continuous Pose Recovery of Human Hands Using Convolutional Networks , 2014, ACM Trans. Graph..

[11]  Chen Qian,et al.  Realtime and Robust Hand Tracking from Depth , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  S. Buss.  Introduction to Inverse Kinematics with Jacobian Transpose , Pseudoinverse and Damped Least Squares methods , 2004 .