Real-Time Optimal Control for Spacecraft Orbit Transfer via Multiscale Deep Neural Networks

This study is motivated by the requirement of on-board trajectory optimization with guaranteed convergence and real-time performance for optimal spacecraft orbit transfers. To this end, a real-time optimal control approach is proposed using deep learning technologies to obtain minimum-time trajectories of solar sail spacecraft for orbit transfer missions. First, the minimum-time two-dimensional orbit transfer problem is solved by an indirect method, and the costate normalization technique is introduced to increase the probability of finding the optimal solutions. Second, by making novel use of deep learning technologies, three deep neural networks are designed and trained offline by the obtained optimal solutions to generate the guidance commands in real time during flight. Consequently, the long-standing difficulty of on-board trajectory generation is resolved. Then, an interactive network training strategy is presented to increase the success rate of finding optima by supplying good initial guesses for the indirect method. Moreover, a multiscale network cooperation strategy is designed to deal with the recognition deficiency of deep neural networks (DNNs) with small input values, which helps achieve highly precise control of terminal orbit insertion. Numerical simulations are given to substantiate the efficiency of these techniques, and illustrate the effectiveness and robustness of the proposed DNN-based trajectory control for future on-board applications.

[1]  Michael J. Grant,et al.  Constrained Trajectory Optimization for Planetary Entry via Sequential Convex Programming , 2016 .

[2]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[3]  Lincoln J. Wood,et al.  Comment on 'Time-optimal orbit transfer trajectory for solar sail spacecraft' , 1982 .

[4]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[5]  Shijie Xu,et al.  Neural-network-based terminal sliding-mode control for thrust regulation of a tethered space-tug , 2018, Astrodynamics.

[6]  Dario Izzo,et al.  Real-time optimal control via Deep Neural Networks: study on landing problems , 2016, ArXiv.

[7]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[8]  Ping Lu,et al.  Entry Guidance: A Unified Method , 2014 .

[9]  Konrad P. Körding,et al.  Deep networks for motor control functions , 2015, Front. Comput. Neurosci..

[10]  Vincent Martinot,et al.  An Integrated Tool for Low Thrust Optimal Control Orbit Transfers in Interplanetary Trajectories , 2004 .

[11]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[12]  Zoran Popovic,et al.  Interactive Control of Diverse Complex Characters with Neural Networks , 2015, NIPS.

[13]  Michael J. Grant,et al.  Autonomous Entry Guidance for Hypersonic Vehicles by Convex Optimization , 2018, Journal of Spacecraft and Rockets.

[14]  Bernd Dachwald,et al.  Optimization of interplanetary solar sailcraft trajectories using evolutionary neurocontrol , 2004 .

[15]  Nan Li,et al.  Co-state initialization for the minimum-time low-thrust trajectory optimization , 2017 .

[16]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[17]  M. Ceriotti,et al.  Solar Sail Trajectory Design for a Multiple Near-Earth Asteroid Rendezvous Mission , 2016 .

[18]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[19]  N. Ansari,et al.  Adaptive fusion by reinforcement learning for distributed detection systems , 1996, IEEE Transactions on Aerospace and Electronic Systems.

[20]  Jinlong Wu,et al.  Data-Driven Augmentation of Turbulence Models with Physics-Informed Machine Learning , 2018 .

[21]  Raffaello D'Andrea,et al.  Performance benchmarking of quadrotor systems using time-optimal control , 2012, Auton. Robots.

[22]  Xi Chen,et al.  Evolution Strategies as a Scalable Alternative to Reinforcement Learning , 2017, ArXiv.

[23]  Sebastian Ruder,et al.  An overview of gradient descent optimization algorithms , 2016, Vestnik komp'iuternykh i informatsionnykh tekhnologii.

[24]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[25]  J. Betts Survey of Numerical Methods for Trajectory Optimization , 1998 .

[26]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[27]  Sohrab Effati,et al.  Optimal control problem via neural networks , 2013, Neural Computing and Applications.

[28]  Farzad Pourboghrat,et al.  Optimal control of nonlinear systems using RBF neural network and adaptive extended Kalman filter , 2009, 2009 American Control Conference.

[29]  David G. Hull,et al.  Optimal Control Theory for Applications , 2003 .

[30]  W. Marsden I and J , 2012 .

[31]  Jorge J. Moré,et al.  User guide for MINPACK-1. [In FORTRAN] , 1980 .

[32]  Xiangdong Liu,et al.  Searching for Time Optimal Periodic Orbits Near Irregularly Shaped Asteroids by Using an Indirect Method , 2017, IEEE Transactions on Aerospace and Electronic Systems.

[33]  Ben Tse,et al.  Autonomous Inverted Helicopter Flight via Reinforcement Learning , 2004, ISER.

[34]  Michael J. Grant,et al.  Minimum-Fuel Low-Thrust Transfers for Spacecraft: A Convex Approach , 2018, IEEE Transactions on Aerospace and Electronic Systems.

[35]  F. Topputo,et al.  Survey of Direct Transcription for Low-Thrust Space Trajectory Optimization with Applications , 2014 .

[36]  Jongho Shin Adaptive Dynamic Surface Control for a Hypersonic Aircraft Using Neural Networks , 2017, IEEE Transactions on Aerospace and Electronic Systems.

[37]  Junfeng Li,et al.  Pseudospectral Methods for Trajectory Optimization with Interior Point Constraints: Verification and Applications , 2013, IEEE Transactions on Aerospace and Electronic Systems.

[38]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.