Haptic Device Design and Teleoperation Control Algorithms for Mobile Manipulators

The increasing need of teleoperated robotic systems implies more and more often to use, as slave devices, mobile platforms (terrestrial, aerial or underwater) with integrated manipulation capabilities, provided e.g. by robotic arms with proper grasping/manipulation tools. Despite this, the research activity in teleoperation of robotic systems has mainly focused on the control of either fixed-base manipulators or mobile robots, non considering the integration of these two types of systems in a single device. Such a combined robotic devices are usually referred to as mobile manipulators: systems composed by both a robotic manipulator and a mobile platform (on which the arm is mounted) whose purpose is to enlarge the manipulator’s workspace. The combination of a mobile platform and a serial manipulator creates redundancy: a particular point in the space can be reached by moving the manipulator, by moving the mobile platform, or by a combined motion of both. A synchronized motion of both devices need then to be addressed. Although specific haptic devices explicitly oriented to the control of mobile manipulators need to be designed, there are no commercial solution yet. For this reason it is often necessary to control such as combined systems with traditional haptic devices not specifically oriented to the control of mobile manipulators. The research activity presented in this Ph.D. thesis focuses in the first place on the design of a teleoperation control scheme which allows the simultaneous control of both the manipulator and the mobile platform by means of a single haptic device characterized by fixed base and an open kinematic chain. Secondly the design of a novel cable-drive haptic devices has been faced. Investigating the use of twisted strings actuation in force rendering is the most interesting challenge of the latter activity.

[1]  Gerd Hirzinger,et al.  ROTEX-the first remotely controlled robot in space , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[2]  John R. Wright,et al.  Driving on Mars with RSVP , 2006, IEEE Robotics & Automation Magazine.

[3]  Benjamin Pitzer,et al.  Towards perceptual shared autonomy for robotic mobile manipulation , 2011, 2011 IEEE International Conference on Robotics and Automation.

[4]  Masaru Uchiyama,et al.  Design of a compact 6-DOF haptic interface , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[5]  Tobias Ortmaier,et al.  Motion estimation in beating heart surgery , 2005, IEEE Transactions on Biomedical Engineering.

[6]  Manuel Brucker,et al.  Virtual reality support for teleoperation using online grasp planning , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  J.-G.S. Demers,et al.  Freedom 6S Force Feedback Hand Controller , 1998 .

[8]  Gerd Hirzinger,et al.  Sensor-based space robotics-ROTEX and its telerobotic features , 1993, IEEE Trans. Robotics Autom..

[9]  Umberto Scarcia.  Design and Control of Robotic Hands , 2015 .

[10]  Takashi Sonoda,et al.  Multi-fingered robotic hand employing strings transmission named “Twist Drive” , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Mark W. Spong,et al.  Bilateral teleoperation: An historical survey , 2006, Autom..

[12]  Robert E. Mahony,et al.  Comparative Study of Haptic Interfaces for Bilateral Teleoperation of VTOL Aerial Robots , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[13]  Robert Riener,et al.  Control aspects of a robotic haptic interface for kinesthetic knee joint simulation , 2002 .

[14]  Thomas B. Sheridan,et al.  One handed tracking in six degrees of freedom , 1989, Conference Proceedings., IEEE International Conference on Systems, Man and Cybernetics.

[15]  Yuru Zhang,et al.  A modified motion mapping method for haptic device based space teleoperation , 2014, The 23rd IEEE International Symposium on Robot and Human Interactive Communication.

[16]  Hugh F. Durrant-Whyte,et al.  Robotic excavation in construction automation , 2002, IEEE Robotics Autom. Mag..

[17]  Tsuneo Yoshikawa,et al.  Bilateral control of master-slave manipulators for ideal kinesthetic coupling-formulation and experiment , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[18]  Richard M. Murray,et al.  A Mathematical Introduction to Robotic Manipulation , 1994 .

[19]  R. Goertz.  FUNDAMENTALS OF GENERAL-PURPOSE REMOTE MANIPULATORS , 1952 .

[20]  Hideki Hashimoto,et al.  Scaled teleoperation system for nano-scale interaction and manipulation , 2003, Adv. Robotics.

[21]  M. Spong,et al.  Robot Modeling and Control , 2005 .

[22]  Claudio Melchiorri,et al.  Teleoperation of a mobile robot through haptic feedback , 2002, IEEE International Workshop HAVE Haptic Virtual Environments and Their.

[23]  Dongjun Lee,et al.  Design and control of a low cost 6 DOF master controller , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[24]  Gianluca Palli,et al.  Experimental Evaluation of Guided Twisted Actuation , 2016 .

[25]  Gerd Hirzinger,et al.  Task directed programming of sensor based robots , 1994, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94).

[26]  Thomas B. Sheridan,et al.  Space teleoperation through time delay: review and prognosis , 1993, IEEE Trans. Robotics Autom..

[27]  Soo S. Lee,et al.  Design of a general purpose 6-DOF haptic interface , 2003 .

[28]  Olav Egeland,et al.  Task-space tracking with redundant manipulators , 1987, IEEE Journal on Robotics and Automation.

[29]  Greg R. Luecke,et al.  Dynamic simulation of virtual mechanisms with haptic feedback using industrial robotics equipment , 1997, Proceedings of International Conference on Robotics and Automation.

[30]  Gianluca Palli,et al.  Integration of an Optical force Sensor into the actuation Module of the DEXMART Hand , 2014, Int. J. Robotics Autom..

[31]  François G. Pin,et al.  Motion planning for mobile manipulators with a non-holonomic constraint using the FSP (full space parameterization) method , 1996 .

[32]  Bernard Bayle,et al.  Manipulability of Wheeled Mobile Manipulators: Application to Motion Generation , 2003 .

[33]  Jeremy A. Marvel,et al.  Survey of Research for Performance Measurement of Mobile Manipulators , 2016 .

[34]  Dale A. Lawrence.  Stability and transparency in bilateral teleoperation , 1993, IEEE Trans. Robotics Autom..

[35]  Dmitry Popov,et al.  A preliminary study on a twisted strings-based elbow exoskeleton , 2013, 2013 World Haptics Conference (WHC).

[36]  Gianluca Palli,et al.  A simple and easy-to-build optoelectronics force sensor based on light fork: Design comparison and experimental evaluation , 2018 .

[37]  Judy M. Vance,et al.  Interacting With Grasped Objects in Expanded Haptic Workspaces Using the Bubble Technique , 2015, J. Comput. Inf. Sci. Eng..

[38]  John F. Gardner,et al.  Kinematics of mobile manipulators and implications for design , 2000, J. Field Robotics.

[39]  Yunyi Jia.  Teleoperation of mobile manipulators , 2014 .

[40]  Bruno Siciliano,et al.  Experimental evaluation of synergy-based in-hand manipulation , 2014 .

[41]  Thomas B. Sheridan,et al.  Human Factors in Automated and Robotic Space Systems: Proceedings of a symposium. Part 1 , 1987 .

[42]  Jae-Bok Song,et al.  Design of a 6 DOF haptic master for teleoperation of a mobile manipulator , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[43]  Jee-Hwan Ryu,et al.  Hybrid position-position and position-speed command strategy for the bilateral teleoperation of a mobile robot , 2007, 2007 International Conference on Control, Automation and Systems.

[44]  Russell H. Taylor,et al.  Constrained Cartesian motion control for teleoperated surgical robots , 1996, IEEE Trans. Robotics Autom..

[45]  Simon Richir,et al.  The "Bubble" technique: interacting with large virtual environments using haptic devices with limited workspace , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[46]  Richard A. Volz,et al.  Tele-autonomous systems: Methods and architectures for intermingling autonomous and telerobotic technology , 1987, Proceedings. 1987 IEEE International Conference on Robotics and Automation.

[47]  Gianluca Palli,et al.  Design and implementation of a simple and low-cost optoelectronic force sensor for robotic applications , 2016, 2016 IEEE International Conference on Advanced Intelligent Mechatronics (AIM).

[48]  Makoto Sato,et al.  A 3D Spatial Interface Device Using Tensed Strings , 1994, Presence: Teleoperators & Virtual Environments.

[49]  Roberto Meattini,et al.  Development of sEMG-driven assistive devices based on twisted string actuation , 2017, 2017 3rd International Conference on Control, Automation and Robotics (ICCAR).

[50]  Uhn Joo Na,et al.  A New 6-DOF Haptic Device for Teleoperation of 6-DOF Serial Robots , 2011, IEEE Transactions on Instrumentation and Measurement.

[51]  Sungchul Kang,et al.  Development of a six DOF haptic master for teleoperation of a mobile manipulator , 2010 .

[52]  Klaus Landzettel,et al.  Robotics Component Verification on ISS ROKVISS - Preliminary Results for Telepresence , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[53]  Roberto Meattini,et al.  A Wearable Robotic Device Based on Twisted String Actuation for Rehabilitation and Assistive Applications , 2017, J. Robotics.

[54]  Claudio Melchiorri,et al.  Control schemes for teleoperation with time delay: A comparative study , 2002, Robotics Auton. Syst..

[55]  J. Marescaux,et al.  Transcontinental Robot-Assisted Remote Telesurgery: Feasibility and Potential Applications , 2002, Annals of surgery.

[56]  Homayoun Seraji,et al.  A Unified Approach to Motion Control of Mobile Manipulators , 1998, Int. J. Robotics Res..

[57]  Robert L. Williams,et al.  Three-Cable Haptic Interface , 2006 .

[58]  R. Uhrich,et al.  Terminus controlled deep ocean manipulator , 1973 .

[59]  Thomas B. Sheridan,et al.  Supervisory control of remote manipulation , 1967, IEEE Spectrum.

[60]  S Stefan Lichiardopol,et al.  A Survey on Teleoperation , 2007 .

[61]  Klaus Landzettel,et al.  Multisensory Shared Autonomy - A Key Issue In The Space Robot Technology Experiment ROTEX , 1992, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems.

[62]  In-Won Park,et al.  Impedance controlled twisted string actuators for tensegrity robots , 2014, 2014 14th International Conference on Control, Automation and Systems (ICCAS 2014).

[63]  C. Natale,et al.  Modeling and Control of the Twisted String Actuation System , 2013, IEEE/ASME Transactions on Mechatronics.

[64]  Bruno Siciliano,et al.  The DEXMART hand: Mechatronic design and experimental evaluation of synergy-based control for human-like grasping , 2014, Int. J. Robotics Res..

[65]  Philippe Coiffet,et al.  Teleoperation and Robotics: Evolution and development , 1985 .

[66]  Ian R. Manchester,et al.  Virtual environment teleoperation of a hydraulic forestry crane , 2008, 2008 IEEE International Conference on Robotics and Automation.

[67]  Thomas B. Sheridan.  Teleoperation, Telerobotics, and Telepresence: A Progress Report , 1992 .

[68]  John Kenneth Salisbury,et al.  The Black Falcon: a teleoperated surgical instrument for minimally invasive surgery , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[69]  Ju-Jang Lee,et al.  Generating artificial force for feedback control of teleoperated mobile robots , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[70]  Thomas H. Massie,et al.  The PHANToM Haptic Interface: A Device for Probing Virtual Objects , 1994 .

[71]  Paolo Gallina,et al.  3-d.o.f. Wire Driven Planar Haptic Interface , 2001, J. Intell. Robotic Syst..

[72]  Samad Hayati,et al.  Design and implementation of a robot control system with traded and shared control capability , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[73]  Lawrence W. Stark,et al.  A comparison of position and rate control for telemanipulations with consideration of manipulator system dynamics , 1987, IEEE Journal on Robotics and Automation.

[74]  Junku Yuh,et al.  Design and Control of Autonomous Underwater Robots: A Survey , 2000, Auton. Robots.

[75]  Klaus Landzettel,et al.  ROKVISS – ROBOTICS COMPONENT VERIFICATION ON ISS , 2005 .

[76]  Ron P. Podhorodeski,et al.  DEXTEROUS TASK-PRIORITY BASED REDUNDANCY RESOLUTION FOR UNDERWATER MANIPULATOR SYSTEMS , 2007 .

[77]  Vincent Hayward,et al.  FREEDOM-7: A High Fidelity Seven Axis Haptic Device with Application to Surgical Training , 1997, ISER.

[78]  T B Sheridan,et al.  Teleoperator Performance with Varying Force and Visual Feedback , 1994, Human factors.

[79]  Sabri Tosunoglu,et al.  A REVIEW OF TELEOPERATION SYSTEM CONTROL , 2003 .

[80]  Nathaniel I. Durlach.  The Potential of Teleoperation for Entertainment and Education , 1997, Presence: Teleoperators & Virtual Environments.

[81]  Davide Chiaravalli,et al.  A hybrid teleoperation control scheme for a single-arm mobile manipulator with omnidirectional wheels , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[82]  Z. Binder,et al.  ANALYSIS OF A COMPUTER-AIDED TELEOPERATION PROCESS BY MEANS OF GENERALIZED STOCHASTIC PETRI NETS , 1997 .

[83]  Blake Hannaford,et al.  Performance evaluation of a six-axis generalized force-reflecting teleoperator , 1991, IEEE Trans. Syst. Man Cybern..

[84]  Tsuneo Yoshikawa,et al.  Ground-space bilateral teleoperation of ETS-VII robot arm by direct bilateral coupling under 7-s time delay condition , 2004, IEEE Transactions on Robotics and Automation.

[85]  Sandra Hirche,et al.  Bilateral Control Architectures for Telerobotics , 2007, Advances in Telerobotics.

[86]  Claudio Melchiorri,et al.  WireMan: a portable wire manipulator for touch-rendering of bas-relief virtual surfaces , 1997, 1997 8th International Conference on Advanced Robotics. Proceedings. ICAR'97.

[87]  Kevin L. Moore,et al.  Optimizing human-robot teleoperation interfaces for mobile manipulators , 2013, Ind. Robot.

[88]  Aude Bolopion,et al.  A Review of Haptic Feedback Teleoperation Systems for Micromanipulation and Microassembly , 2013, IEEE Transactions on Automation Science and Engineering.

[89]  Delbert Tesar,et al.  Construction and demonstration of a 9-string 6 DOF force reflecting joystick for telerobotics , 1989 .

[90]  Claudio Melchiorri.  Robotic telemanipulation systems: an overview on control aspects , 2003 .

[91]  Jorge Barrio,et al.  A rate-position haptic controller for large telemanipulation workspaces , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[92]  Giuseppe Oriolo,et al.  Kinematic modeling and redundancy resolution for nonholonomic mobile manipulators , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[93]  Martin Buss,et al.  Design, Control, and Evaluation of a Hyper-redundant Haptic Device , 2007, Advances in Telerobotics.

[94]  Henry Y. K. Lau,et al.  Implementation of position–force and position–position teleoperator controllers with cable-driven mechanisms , 2005 .

[95]  Yaohong Qu,et al.  Environmental Force Reflection In An Admittance Configured Haptic Interface For Teleoperation Of VTOL Aerial Robots , 2017 .

[96]  Alexander Dietrich,et al.  Singularity avoidance for nonholonomic, omnidirectional wheeled mobile platforms with variable footprint , 2011, 2011 IEEE International Conference on Robotics and Automation.

[97]  Costas S. Tzafestas,et al.  Visuo-Haptic Interface for Teleoperation of Mobile Robot Exploration Tasks , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[98]  Gianluca Palli,et al.  Development of an haptic interface based on twisted string actuators , 2017, 2017 IEEE International Conference on Advanced Intelligent Mechatronics (AIM).

[99]  Ignacy Duleba,et al.  Modeling and Control of Mobile Manipulators , 2000 .

[100]  Jindong Tan,et al.  Integrated Task Planning and Control for Mobile Manipulators , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).