A novel human-robot interface using hybrid sensors with Kalman filters

Purpose – The aim of this paper is to present a novel methodology which incorporates Camshift, Kalman filter (KFs) and adaptive multi-space transformation (AMT) for a human-robot interface, which perfects human intelligence and teleoperation. Design/methodology/approach – In the proposed method, an inertial measurement unit is used to measure the orientation of the human hand, and a Camshift algorithm is used to track the human hand using a three-dimensional camera. Although the location and the orientation of the human can be obtained from the two sensors, the measurement error increases over time due to the noise of the devices and the tracking errors. KFs are used to estimate the location and the orientation of the human hand. Moreover, to be subject to the perceptive limitations and the motor limitations, human operator is hard to carry out the high precision operation. An AMT method is proposed to assist the operator to improve accuracy and reliability in determining the pose of the robot. Findings –...

[1]  Robert B. McGhee,et al.  A Simplified Quaternion-Based Algorithm for Orientation Estimation From Earth Gravity and Magnetic Field Measurements , 2008, IEEE Transactions on Instrumentation and Measurement.

[2]  Jonathan Kofman,et al.  Teleoperation of a robot manipulator using a vision-based human-robot interface , 2005, IEEE Transactions on Industrial Electronics.

[3]  Ping Zhang,et al.  Human–Manipulator Interface Based on Multisensory Process via Kalman Filters , 2014, IEEE Transactions on Industrial Electronics.

[4]  Moonhong Baeg,et al.  Spatial Uncertainty Model for Visual Features Using a Kinect™ Sensor , 2012, Sensors.

[5]  Martin Buss,et al.  Influence of Varied Human Movement Control on Task Performance and Feeling of Telepresence , 2010, PRESENCE: Teleoperators and Virtual Environments.

[6]  Terrence Fong,et al.  Novel interfaces for remote driving: gesture, haptic, and PDA , 2001, SPIE Optics East.

[7]  Farid Golnaraghi,et al.  A Fastening Tool Tracking System Using an IMU and a Position Sensor With Kalman Filters and a Fuzzy Expert System , 2009, IEEE Transactions on Industrial Electronics.

[8]  Christian Smith,et al.  Robot manipulators , 2009, IEEE Robotics & Automation Magazine.

[9]  Nor Ashidi Mat Isa,et al.  Enhanced moving K-means (EMKM) algorithm for image segmentation , 2011, IEEE Transactions on Consumer Electronics.

[10]  Patrick Lambert,et al.  Dynamic Hand Gesture Recognition Using the Skeleton of the Hand , 2005, EURASIP J. Adv. Signal Process..

[11]  Takeshi Ando,et al.  A Haptic Interface “Force Blinker 2” for Navigation of the Visually Impaired , 2012, IEEE Transactions on Industrial Electronics.

[12]  Du Guanglong,et al.  Markerless Kinect-Based Hand Tracking for Robot Teleoperation , 2012 .

[13]  Keum-Bae Cho,et al.  Intelligent Lead: A Novel HRI Sensor for Guide Robots , 2012, Sensors.

[14]  Javier Ruiz Hidalgo,et al.  Real-Time Head and Hand Tracking Based on 2.5D Data , 2012 .

[15]  Nicolas D. Georganas,et al.  Real-Time Hand Gesture Detection and Recognition Using Bag-of-Features and Support Vector Machine Techniques , 2011, IEEE Transactions on Instrumentation and Measurement.

[16]  M. Press.  Presence : teleoperators and virtual environments. , 2014 .

[17]  Ping Zhang,et al.  Human–manipulator interface using hybrid sensors with Kalman filters and adaptive multi-space transformation , 2014 .

[18]  Tsukasa Ogasawara,et al.  A hand-pose estimation for vision-based human interfaces , 2003, IEEE Trans. Ind. Electron..

[19]  Yiannis Demiris,et al.  Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction , 2008, HRI 2008.

[20]  Jianwei Zhang,et al.  A two-arm situated artificial communicator for human-robot cooperative assembly , 2003, IEEE Trans. Ind. Electron..

[21]  Che-Hao Chang,et al.  Improved Hand Tracking System , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Fernando Torres Medina,et al.  Hybrid tracking of human operators using IMU/UWB data fusion by a Kalman filter , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[23]  Sandra Hirche,et al.  Human-Oriented Control for Haptic Teleoperation , 2012, Proceedings of the IEEE.

[24]  Guanglong Du,et al.  Human-Manipulator Interface Using Particle Filter , 2014, TheScientificWorldJournal.

[25]  W. Singhose,et al.  Hand-Motion Crane Control Using Radio-Frequency Real-Time Location Systems , 2012, IEEE/ASME Transactions on Mechatronics.

[26]  Jonathan Kofman,et al.  Robot-Manipulator Teleoperation by Markerless Vision-Based Hand-Arm Tracking , 2007 .

[27]  Kiyoshi Ohishi,et al.  Force Control of Human–Robot Interaction Using Twin Direct-Drive Motor System Based on Modal Space Design , 2010, IEEE Transactions on Industrial Electronics.

[28]  Takayuki Kanda,et al.  Teleoperation of Multiple Social Robots , 2012, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[29]  Raul Wirz,et al.  A multimodal interface to control a robot arm via the web: a case study on remote programming , 2005, IEEE Transactions on Industrial Electronics.

[30]  Mehran Jahed,et al.  A Neuro–Fuzzy Inference System for sEMG-Based Identification of Hand Motion Commands , 2011, IEEE Transactions on Industrial Electronics.

[31]  Toshio Fukuda,et al.  An exoskeletal robot for human elbow motion support-sensor fusion, adaptation, and control , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[32]  N. Alberto Borghese,et al.  Tracking densely moving markers , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[33]  Luis M. Muñoz,et al.  Improving the Human–Robot Interface Through Adaptive Multispace Transformation , 2009, IEEE Transactions on Robotics.

[34]  C. Poole,et al.  Classical Mechanics, 3rd ed. , 2002 .

[35]  W. Murray.  Numerical Methods for Unconstrained Optimization , 1975 .