Low-Thrust Trajectory Design Using Closed-Loop Feedback-Driven Control Laws and State-Dependent Parameters

Low-thrust many-revolution trajectory design and orbit transfers are becoming increasingly important with the development of high specific impulse, low-thrust engines. Closed-loop feedback-driven (CLFD) control laws can be used to solve these trajectory design problems with minimal computational cost and offer potential for autonomous guidance. However, they have user-defined parameters which limit their optimality. In this work, an actor-critic reinforcement learning framework is proposed to make the parameters of the Lyapunov-based Q-law state-dependent, ensuring the controller can adapt as the dynamics evolve during a transfer. The proposed framework should be independent of the particular CLFD control law and provides improved solutions for mission analysis. There is also potential for future on-board autonomous use, as trajectories are closed-form and can be generated without an initial guess. The current results focus on GTO-GEO transfers in Keplerian dynamics and later with eclipse and J2 effects. Both time-optimal and mass-optimal transfers are presented, and the stability to uncertainties in orbit determination are discussed. The task of handling orbit perturbations is left to future work.

[1]  Yang Gao,et al.  Optimization of low-thrust many-revolution transfers and Lyapunov-based guidance , 2010 .

[2]  Robert Babuska,et al.  A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[3]  Bruce A. Conway,et al.  Spacecraft Trajectory Optimization: Contents , 2010 .

[4]  Jose A. Lozano,et al.  Spacecraft trajectory optimization: A review of models, objectives, approaches and solutions , 2018, Progress in Aerospace Sciences.

[5]  Mauro Massari,et al.  Adaptive Generalized ZEM-ZEV Feedback Guidance for Planetary Landing via a Deep Reinforcement Learning Approach , 2020, ArXiv.

[6]  David A. Smith,et al.  Comparison of Low-Thrust Control Laws for Application in Planetocentric Space , 2014 .

[7]  Giorgio Fasano,et al.  Modeling and Optimization in Space Engineering , 2012, Springer Optimization and Its Applications.

[8]  Guang-Bin Huang,et al.  What are Extreme Learning Machines? Filling the Gap Between Frank Rosenblatt’s Dream and John von Neumann’s Puzzle , 2015, Cognitive Computation.

[9]  Lei Zhang,et al.  Optimal low-thrust spiral trajectories using Lyapunov-based guidance , 2016 .

[10]  Craig A. Kluever,et al.  Simple Guidance Scheme for Low-Thrust Orbit Transfers , 1998 .

[11]  Benjamin E. Joseph.  Lyapunov feedback control in equinoctial elements applied to low thrust control of elliptical orbit constellations , 2006 .

[12]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[13]  D. Pérez-Palau,et al.  Lyapunov-based low-energy low-thrust transfers to the Moon , 2019, Acta Astronautica.

[14]  Richard Epenoy,et al.  Optimal low-thrust transfers with constraints---generalization of averaging techniques , 1997 .