Reinforcement Learning-Based Minimum Energy Position Control of Dielectric Elastomer Actuators

This article deals with the closed-loop optimal control of mechatronic devices based on dielectric elastomer membranes. The goal is to minimize the input electrical energy required to achieve a given position regulation task. The actuator is modeled based on a free-energy framework, which provides a thermodynamically consistent characterization of the losses that occur during actuation. Due to the strongly nonlinear behavior of both system model and dissipation function, traditional techniques based on the analytical solution of the Hamilton–Jacobi–Bellman (HJB) equation cannot be applied. Therefore, a reinforcement learning-based algorithm is here proposed as a tool to solve, offline, the HJB equation related to the energy minimization problem. After discussing the theory, experimental results are presented to validate the effectiveness of the proposed approach for different positioning tasks.

[1]  Frank L. Lewis,et al.  $ {H}_{ {\infty }}$ Tracking Control of Completely Unknown Continuous-Time Systems via Off-Policy Reinforcement Learning , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[2]  D. De Rossi,et al.  Electroactive Elastomeric Actuator for All-Polymer Linear Peristaltic Pumps , 2010, IEEE/ASME Transactions on Mechatronics.

[3]  Jiang Zou,et al.  Feedforward Control of the Rate-Dependent Viscoelastic Hysteresis Nonlinearity in Dielectric Elastomer Actuators , 2019, IEEE Robotics and Automation Letters.

[4]  P. McHugh,et al.  A review on dielectric elastomer actuators, technology, applications, and challenges , 2008 .

[5]  Jiang Zou,et al.  High-Precision Tracking Control of a Soft Dielectric Elastomer Actuator With Inverse Viscoelastic Hysteresis Compensation , 2019, IEEE/ASME Transactions on Mechatronics.

[6]  Rocco Vertechy,et al.  Advances in the development of dielectric elastomer generators for wave energy conversion , 2020, Renewable and Sustainable Energy Reviews.

[7]  Stefan Seelecke,et al.  Modeling, Identification, and Control of a Dielectric Electro-Active Polymer Positioning System , 2015, IEEE Transactions on Control Systems Technology.

[8]  Bui Ngoc Minh Truong,et al.  Inverse modeling and control of a dielectric electro-active polymer smart actuator , 2015 .

[9]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[10]  Frank L. Lewis,et al.  Off-Policy Actor-Critic Structure for Optimal Control of Unknown Systems With Disturbances , 2016, IEEE Transactions on Cybernetics.

[11]  Stefan Seelecke,et al.  Experimental characterization of the hysteretic and rate-dependent electromechanical behavior of dielectric electro-active polymer actuators , 2010 .

[12]  Stefan Seelecke,et al.  Towards Sensorless Soft Robotics: Self-Sensing Stiffness Control of Dielectric Elastomer Actuators , 2020, IEEE Transactions on Robotics.

[13]  Xiangyang Zhu,et al.  Automatic Design of Soft Dielectric Elastomer Actuators With Optimal Spatial Electric Fields , 2019, IEEE Transactions on Robotics.

[14]  Frank L. Lewis,et al.  Adaptive Optimal Control of Unknown Constrained-Input Systems Using Policy Iteration and Neural Networks , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[15]  G. Kovács,et al.  Dielectric elastomer actuators used for pneumatic valve technology , 2013 .

[16]  Huaguang Zhang,et al.  Adaptive Dynamic Programming: An Introduction , 2009, IEEE Computational Intelligence Magazine.

[17]  Tareq Assaf,et al.  Cerebellar-inspired algorithm for adaptive control of nonlinear dielectric elastomer-based artificial muscle , 2016, Journal of The Royal Society Interface.

[18]  Rahimullah Sarban,et al.  Physical model-based active vibration control using a dielectric elastomer actuator , 2012 .

[19]  Xiangyang Zhu,et al.  A survey on dielectric elastomer actuators for soft robots , 2017, Bioinspiration & biomimetics.

[20]  Frank L. Lewis,et al.  Online actor critic algorithm to solve the continuous-time infinite horizon optimal control problem , 2009, 2009 International Joint Conference on Neural Networks.

[21]  Wenyu Liang,et al.  Control of Dielectric Elastomer Soft Actuators Using Antagonistic Pairs , 2019, IEEE/ASME Transactions on Mechatronics.

[22]  Elaine Biddiss,et al.  Dielectric elastomers as actuators for upper limb prosthetics: challenges and opportunities. , 2008, Medical engineering & physics.

[23]  Frank L. Lewis,et al.  Adaptive optimal control for continuous-time linear systems based on policy iteration , 2009, Autom..

[24]  W Kaal,et al.  Electroactive Polymer Actuators in Dynamic Applications , 2011, IEEE/ASME Transactions on Mechatronics.

[25]  Hoa Phung,et al.  Multiple-Degrees-of-Freedom Dielectric Elastomer Actuators for Soft Printable Hexapod Robot , 2017 .

[26]  Gaël Chevallier,et al.  A new identification method of viscoelastic behavior: Application to the generalized Maxwell model , 2011 .

[27]  Thorben Hoffstadt,et al.  Adaptive Sliding-Mode Position Control for Dielectric Elastomer Actuators , 2017, IEEE/ASME Transactions on Mechatronics.

[28]  Zhong-Ping Jiang,et al.  Robust Adaptive Dynamic Programming , 2017 .

[29]  G Jordan,et al.  Actuated Micro-optical Submount Using a Dielectric Elastomer Actuator , 2011, IEEE/ASME Transactions on Mechatronics.

[30]  Stefan Seelecke,et al.  A Thermodynamically Consistent Port-Hamiltonian Model for Dielectric Elastomer Membrane Actuators and Generators , 2017 .

[31]  Huai-Ning Wu,et al.  Policy Gradient Adaptive Dynamic Programming for Data-Based Optimal Control , 2017, IEEE Transactions on Cybernetics.

[32]  Xiangyang Zhu,et al.  Modeling of Viscoelastic Electromechanical Behavior in a Soft Dielectric Elastomer Actuator , 2017, IEEE Transactions on Robotics.

[33]  D. Floreano,et al.  Versatile Soft Grippers with Intrinsic Electroadhesion Based on Multifunctional Polymer Actuators , 2016, Advanced materials.

[34]  K. Bertoldi,et al.  Dielectric Elastomer Based “Grippers” for Soft Robotics , 2015, Advanced materials.

[35]  P Lotz,et al.  Fabrication and Application of Miniaturized Dielectric Elastomer Stack Actuators , 2011, IEEE/ASME Transactions on Mechatronics.

[36]  Stefan Seelecke,et al.  Robust Position Control of Dielectric Elastomer Actuators Based on LMI Optimization , 2016, IEEE Transactions on Control Systems Technology.

[37]  J. Swinburne Electromagnetic Theory , 1894, Nature.

[38]  Stefan Seelecke,et al.  Robust LMI position regulation of a bistable Dielectric Electro-Active Polymer membrane , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[39]  F.L. Lewis,et al.  Reinforcement learning and adaptive dynamic programming for feedback control , 2009, IEEE Circuits and Systems Magazine.

[40]  Gianluca Rizzello,et al.  Energy Optimal Control of Dielectric Elastomer Actuators via Adaptive Dynamic Programming , 2019 .

[41]  Xiangyang Zhu,et al.  Feedforward deformation control of a dielectric elastomer actuator based on a nonlinear dynamic model , 2015 .