Continuous Control of a Soft Continuum Arm using Deep Reinforcement Learning

Soft Continuum Arms (SCAs) are challenging to control due to their highly nonlinear characteristics and sensitivity to external loading. Recent efforts to address the control problem using machine learning techniques are limited to simple SCA architectures. In this paper, we train a model-free reinforcement learning control policy based on Deep Deterministic Policy Gradient (DDPG) for end effector path tracking on a BR2 SCA. Unlike simple SCA architectures, the BR2 SCA has the functionality to bend and rotate spatially thus leading to enhanced workspace and ability to perform complex tasks. The control policy is first validated in simulations and then implemented on a prototype BR2 with state feedback. An average tracking error less than 3 cm (< diameter of the SCA) is reported using the proposed control policy. The efficacy of the control policy is validated for different loading conditions both in simulations and on the SCA prototype.

[1]  Cecilia Laschi,et al.  Model-Based Reinforcement Learning for Closed-Loop Dynamic Control of Soft Robotic Manipulators , 2019, IEEE Transactions on Robotics.

[2]  Sergey Levine,et al.  Temporal Difference Models: Model-Free Deep RL for Model-Based Control , 2018, ICLR.

[3]  Kevin C. Galloway,et al.  A Dexterous, Glove-Based Teleoperable Low-Power Soft Robotic Arm for Delicate Deep-Sea Biological Exploration , 2018, Scientific Reports.

[4]  Mariangela Manti,et al.  Multiobjective Optimization for Stiffness and Position Control in a Soft Robot Arm Module , 2018, IEEE Robotics and Automation Letters.

[5]  Ian D. Walker,et al.  OctArm - A soft robotic manipulator , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[6]  Cecilia Laschi,et al.  Learning dynamic models for open loop predictive control of soft robotic manipulators. , 2017, Bioinspiration & biomimetics.

[7]  Mariangela Manti,et al.  Exploiting Morphology of a Soft Manipulator for Assistive Tasks , 2017, Living Machines.

[8]  Gaurav Singh,et al.  Parameter estimation and modeling of a pneumatic continuum manipulator with asymmetric building blocks , 2018, 2018 IEEE International Conference on Soft Robotics (RoboSoft).

[9]  Girish Krishnan,et al.  Towards Pneumatic Spiral Grippers: Modeling and Design Considerations. , 2018, Soft robotics.

[10]  Hao Jiang,et al.  Model-free control for soft manipulators based on reinforcement learning , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[11]  Stefan Schaal,et al.  Learning inverse kinematics , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[12]  Michael I. Jordan,et al.  Reinforcement Learning Algorithm for Partially Observable Markov Decision Problems , 1994, NIPS.

[13]  Girish Chowdhary,et al.  Open Loop Position Control of Soft Continuum Arm Using Deep Reinforcement Learning , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[14]  CianchettiMatteo,et al.  A Bioinspired Soft Robotic Gripper for Adaptable and Effective Grasping , 2015 .

[15]  Feng Wu,et al.  Toward Effective Soft Robot Control via Reinforcement Learning , 2017, ICIRA.

[16]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[17]  Pieter Abbeel,et al.  Predicting initialization effectiveness for trajectory optimization , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Girish Krishnan,et al.  VaLeNS: Design of a Novel Variable Length Nested Soft Arm , 2020, IEEE Robotics and Automation Letters.

[19]  Isuru S. Godage,et al.  Validation of an Extensible Rod Model for Soft continuum Manipulators , 2019, 2019 2nd IEEE International Conference on Soft Robotics (RoboSoft).

[20]  Michael C. Yip,et al.  Model-Less Feedback Control of Continuum Manipulators in Constrained Environments , 2014, IEEE Transactions on Robotics.

[21]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.