Real-time hand guiding of industrial manipulator in 5 DOF using Microsoft Kinect and accelerometer

Human computer interaction is currently developing toward more intuitive and natural ways of communication. The main goal of research presented in this paper is the development of a system for controlling a robot manipulator in 5 DOF using human motions. The developed system reads and interprets sensor data from a Microsoft Kinect and an accelerometer embedded in a smartphone and use this data and the robot kinematics to generate a position and orientation reference to the robot controller allowing real-time interaction with the robot. The system has been implemented and tested on a Universal Robot manipulator arm with 6 joints. This robot has an embedded controller able to receive a reference in both Cartesian and joint space and calculate and track trajectories that fulfills this reference. The main result of this research is the architecture of an industrially oriented hand guiding system.

[1]  Yael Edan,et al.  Vision-based hand-gesture applications , 2011, Commun. ACM.

[2]  Robert LeMoyne,et al.  Implementation of an iPhone for characterizing Parkinson's disease tremor through a wireless accelerometer application , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[3]  Ankit Chaudhary,et al.  Tracking of Fingertips and Centers of Palm Using KINECT , 2011, 2011 Third International Conference on Computational Intelligence, Modelling & Simulation.

[4]  François Chaumette,et al.  Visual servo control. I. Basic approaches , 2006, IEEE Robotics & Automation Magazine.

[5]  Adam Rogowski,et al.  Industrially oriented voice control system , 2012 .

[6]  Simon A. Neild,et al.  Estimation of Upper-Limb Orientation Based on Accelerometer and Gyroscope Measurements , 2008, IEEE Transactions on Biomedical Engineering.

[7]  Kanad K. Biswas,et al.  Gesture recognition using Microsoft Kinect® , 2011, The 5th International Conference on Automation, Robotics and Applications.

[8]  Jorge Pomares,et al.  Visual Control of Robots Using Range Images , 2010, Sensors.

[9]  Gordon Cheng,et al.  Accelerometer based robotic joint orientation estimation , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[10]  Luc Van Gool,et al.  Real-time 3D hand gesture interaction with a robot for understanding directions from humans , 2011, 2011 RO-MAN.

[11]  Tilak Dutta,et al.  Evaluation of the Kinect™ sensor for 3-D kinematic measurement in the workplace. , 2012, Applied ergonomics.

[12]  Chan-Soo Park,et al.  Comparison of plane extraction performance using laser scanner and Kinect , 2011, 2011 8th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).

[13]  Dugki Min,et al.  Behavioral synchronization of human and humanoid robot , 2011, 2011 8th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).

[14]  Jonathan W. Decker,et al.  Performance measurements for the Microsoft Kinect skeleton , 2012, 2012 IEEE Virtual Reality Workshops (VRW).