Attractor Selection in Nonlinear Energy Harvesting Using Deep Reinforcement Learning

Recent research efforts demonstrate that the intentional use of nonlinearity enhances the capabilities of energy harvesting systems. One of the primary challenges that arise in nonlinear harvesters is that nonlinearities can often result in multiple attractors with both desirable and undesirable responses that may co-exist. This paper presents a nonlinear energy harvester which is based on translation-to-rotational magnetic transmission and exhibits coexisting attractors with different levels of electric power output. In addition, a control method using deep reinforcement learning was proposed to realize attractor switching between coexisting attractors with constrained actuation.

[1]  Mahdi Imani,et al.  Control of Gene Regulatory Networks With Noisy Measurements and Uncertain Inputs , 2017, IEEE Transactions on Control of Network Systems.

[2]  Chung Chen,et al.  A reinforcement learning control scheme for nonlinear systems with multiple actions , 1996, Soft Computing in Intelligent Systems and Information Processing. Proceedings of the 1996 Asian Fuzzy Systems Symposium.

[3]  Daniel J. Inman,et al.  On the optimal energy harvesting from a vibration source using a PZT stack , 2009 .

[4]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[5]  Aniruddha Datta,et al.  External Control in Markovian Genetic Regulatory Networks , 2004, Machine Learning.

[6]  Sergey Levine,et al.  Guided Policy Search , 2013, ICML.

[7]  Long Ji Lin,et al.  Self-improving reactive agents based on reinforcement learning, planning and teaching , 1992, Machine Learning.

[8]  Neil D. Sims,et al.  Energy harvesting from the nonlinear oscillations of magnetic levitation , 2009 .

[9]  D. Inman,et al.  Nonlinear piezoelectricity in electroelastic energy harvesters: Modeling and experimental identification , 2010 .

[10]  Chee Kiong Soh,et al.  Broadband Vibration Energy Harvesting Techniques , 2013 .

[11]  R.J. Williams,et al.  Reinforcement learning is direct adaptive optimal control , 1991, IEEE Control Systems.

[12]  Ephrahim Garcia,et al.  Piezoelectric resonance shifting using tunable nonlinear stiffness , 2009, Smart Structures and Materials + Nondestructive Evaluation and Health Monitoring.

[13]  Brian P. Mann,et al.  Intentional Nonlinearity in Energy Harvesting Systems , 2019, Understanding Complex Systems.

[14]  Raymond B. Sepe,et al.  Ocean wave energy harvesting buoy for sensors , 2009, 2009 IEEE Energy Conversion Congress and Exposition.

[15]  Yuval Tassa,et al.  MuJoCo: A physics engine for model-based control , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  I. Kovacic,et al.  Potential benefits of a non-linear stiffness in an energy harvesting device , 2010 .

[17]  S. Jo,et al.  Passive-self-tunable vibrational energy harvester , 2011, 2011 16th International Solid-State Sensors, Actuators and Microsystems Conference.

[18]  Charles R. Farrar,et al.  Energy Harvesting for Structural Health Monitoring Sensor Networks , 2008 .

[19]  David P. Arnold,et al.  Spherical, rolling magnet generators for passive energy harvesting from human motion , 2009 .

[20]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[21]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[22]  J A Hoffer,et al.  Biomechanical Energy Harvesting: Generating Electricity During Walking with Minimal User Effort , 2008, Science.

[23]  Yuval Tassa,et al.  Learning Continuous Control Policies by Stochastic Value Gradients , 2015, NIPS.

[24]  K. W. Wang,et al.  Energy Harvester Synthesis Via Coupled Linear-Bistable System With Multistable Dynamics , 2014 .

[25]  Jun Nakanishi,et al.  Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors , 2013, Neural Computation.

[26]  Xiaogan Li,et al.  Multifunctional TENG for Blue Energy Scavenging and Self‐Powered Wind‐Speed Sensor , 2017 .

[27]  Yiannos Manoli,et al.  A closed-loop wide-range tunable mechanical resonator for energy harvesting systems , 2009 .

[28]  Sotiris Moschoyiannis,et al.  Deep Reinforcement Learning for Control of Probabilistic Boolean Networks , 2019, ArXiv.

[29]  D. Dane Quinn,et al.  The Effect of Non-linear Piezoelectric Coupling on Vibration-based Energy Harvesting , 2009 .

[30]  Brian P. Mann,et al.  Dynamics of a Magnetically Excited Rotational System , 2019, Nonlinear Structures and Systems, Volume 1.

[31]  James Bergstra,et al.  Benchmarking Reinforcement Learning Algorithms on Real-World Robots , 2018, CoRL.

[32]  Sergey Levine,et al.  High-Dimensional Continuous Control Using Generalized Advantage Estimation , 2015, ICLR.

[33]  Martin A. Riedmiller,et al.  Autonomous reinforcement learning on raw visual input data in a real world application , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[34]  James D. Turner,et al.  Reinforcement Learning for Active Damping of Harmonically Excited Pendulum with Highly Nonlinear Actuator , 2019, Nonlinear Structures and Systems, Volume 1.

[35]  D. Inman,et al.  A piezomagnetoelastic structure for broadband vibration energy harvesting , 2009 .

[36]  Khalil Najafi,et al.  Harvesting traffic-induced vibrations for structural health monitoring of bridges , 2011 .

[37]  James D. Turner,et al.  A Model-Free Sampling Method for Estimating Basins of Attraction Using Hybrid Active Learning (HAL) , 2020 .

[38]  Brian P. Mann,et al.  Uncertainty in performance for linear and nonlinear energy harvesting strategies , 2012 .

[39]  Yang Zhang,et al.  Toward self-tuning adaptive vibration-based microgenerators , 2005, SPIE Micro + Nano Materials, Devices, and Applications.

[40]  Reda Alhajj,et al.  Employing Batch Reinforcement Learning to Control Gene Regulation Without Explicitly Constructing Gene Regulatory Networks , 2013, IJCAI.

[41]  Brian P. Mann,et al.  Investigations of a nonlinear energy harvester with a bistable potential well , 2010 .

[42]  D. Inman,et al.  Frequency Self-tuning Scheme for Broadband Vibration Energy Harvesting , 2010 .

[43]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[44]  James D. Turner,et al.  Constrained attractor selection using deep reinforcement learning , 2019, Journal of Vibration and Control.

[45]  Brian P. Mann,et al.  Nonlinear dynamics of a non-contact translational-to-rotational magnetic transmission , 2019, Journal of Sound and Vibration.

[46]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[47]  Benjamin A. M. Owens,et al.  Melnikov theoretic methods for characterizing the dynamics of the bistable piezoelectric inertial generator in complex spectral environments , 2012 .

[48]  Pedro Dinis Gaspar,et al.  Review and Future Trend of Energy Harvesting Methods for Portable Medical Devices , 2010 .

[49]  B. Mann,et al.  Reversible hysteresis for broadband magnetopiezoelastic energy harvesting , 2009 .

[50]  Dibin Zhu,et al.  CLOSED LOOP FREQUENCY TUNING OF A VIBRATION-BASED MICRO- GENERATOR , 2008 .