Model-Based Reinforcement Learning for Closed-Loop Dynamic Control of Soft Robotic Manipulators

Dynamic control of soft robotic manipulators is an open problem yet to be well explored and analyzed. Most of the current applications of soft robotic manipulators utilize static or quasi-dynamic controllers based on kinematic models or linearity in the joint space. However, such approaches are not truly exploiting the rich dynamics of a soft-bodied system. In this paper, we present a model-based policy learning algorithm for closed-loop predictive control of a soft robotic manipulator. The forward dynamic model is represented using a recurrent neural network. The closed-loop policy is derived using trajectory optimization and supervised learning. The approach is verified first on a simulated piecewise constant strain model of a cable driven under-actuated soft manipulator. Furthermore, we experimentally demonstrate on a soft pneumatically actuated manipulator how closed-loop control policies can be derived that can accommodate variable frequency control and unmodeled external loads.

[1]  CianchettiMatteo,et al.  Learning Closed Loop Kinematic Controllers for Continuum Manipulators in Unstructured Environments. , 2017, Soft robotics.

[2]  Roger W. Brockett,et al.  Robotic manipulators and the product of exponentials formula , 1984 .

[3]  Carl E. Rasmussen,et al.  Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning , 2011, Robotics: Science and Systems.

[4]  Oliver Sawodny,et al.  Dynamic Control of the Bionic Handling Assistant , 2017, IEEE/ASME Transactions on Mechatronics.

[5]  Sergey Levine,et al.  Learning compound multi-step controllers under unknown dynamics , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Mariangela Manti,et al.  Soft assistive robot for personal care of elderly people , 2016, 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[7]  Ian D. Walker,et al.  A Neural Network Controller for Continuum Robots , 2007, IEEE Transactions on Robotics.

[8]  Sergey Levine,et al.  Learning deep control policies for autonomous aerial vehicles with MPC-guided policy search , 2015, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[9]  Sergey Levine,et al.  Optimal control with learned local models: Application to dexterous manipulation , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Sergey Levine,et al.  Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics , 2014, NIPS.

[11]  Matteo Cianchetti,et al.  Dynamic Model of a Multibending Soft Robot Arm Driven by Cables , 2014, IEEE Transactions on Robotics.

[12]  Cecilia Laschi,et al.  Learning dynamic models for open loop predictive control of soft robotic manipulators. , 2017, Bioinspiration & biomimetics.

[13]  Matteo Cianchetti,et al.  Soft robotics: Technologies and systems pushing the boundaries of robot abilities , 2016, Science Robotics.

[14]  Ian D. Walker,et al.  Task-space control of extensible continuum manipulators , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Daniela Rus,et al.  Dynamics and trajectory optimization for a soft spatial fluidic elastomer manipulator , 2016, Int. J. Robotics Res..

[16]  Guilherme De A. Barreto,et al.  Long-term time series prediction with the NARX network: An empirical evaluation , 2008, Neurocomputing.

[17]  Mariangela Manti,et al.  Stable Open Loop Control of Soft Robotic Manipulators , 2018, IEEE Robotics and Automation Letters.

[18]  Cecilia Laschi,et al.  Control Strategies for Soft Robotic Manipulators: A Survey. , 2018, Soft robotics.

[19]  Stefan Schaal,et al.  2008 Special Issue: Reinforcement learning of motor skills with policy gradients , 2008 .

[20]  Daniela Rus,et al.  Design, kinematics, and control of a soft spatial fluidic elastomer manipulator , 2016, Int. J. Robotics Res..

[21]  Emanuel Todorov,et al.  Combining the benefits of function approximation and trajectory optimization , 2014, Robotics: Science and Systems.

[22]  Mariangela Manti,et al.  Towards the development of a soft manipulator as an assistive robot for personal care of elderly people , 2017 .

[23]  Lakmal Seneviratne,et al.  Discrete Cosserat Approach for Multisection Soft Manipulator Dynamics , 2017, IEEE Transactions on Robotics.

[24]  Sergey Levine,et al.  Guided Policy Search , 2013, ICML.

[25]  Christopher G. Atkeson,et al.  A comparison of direct and model-based reinforcement learning , 1997, Proceedings of International Conference on Robotics and Automation.

[26]  Lakmal Seneviratne,et al.  Screw-Based Modeling of Soft Manipulators With Tendon and Fluidic Actuation , 2017 .

[27]  Mariangela Manti,et al.  Stiffening in Soft Robotics: A Review of the State of the Art , 2016, IEEE Robotics & Automation Magazine.

[28]  Marc D. Killpack,et al.  A New Soft Robot Control Method: Using Model Predictive Control for a Pneumatically Actuated Humanoid , 2016, IEEE Robotics & Automation Magazine.

[29]  Matteo Bianchi,et al.  Controlling Soft Robots: Balancing Feedback and Feedforward Elements , 2017, IEEE Robotics & Automation Magazine.

[30]  Ian D. Walker,et al.  A model-based sliding mode controller for extensible continuum robots , 2010 .

[31]  Darwin G. Caldwell,et al.  Dynamics for variable length multisection continuum arms , 2016, Int. J. Robotics Res..

[32]  Ian D. Walker,et al.  Empirical investigation of closed-loop control of extensible continuum manipulators , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[33]  Yohan Dupuis,et al.  A Study of Vicon System Positioning Performance , 2017, Sensors.