Mission schedule of agile satellites based on Proximal Policy Optimization Algorithm

Mission schedule of satellites is an important part of space operation nowadays, since the number and types of satellites in orbit are increasing tremendously and their corresponding tasks are also becoming more and more complicated. In this paper, a mission schedule model combined with Proximal Policy Optimization Algorithm(PPO) is proposed. Different from the traditional heuristic planning method, this paper incorporate reinforcement learning algorithms into it and find a new way to describe the problem. Several constraints including data download are considered in this paper.

[1]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[2]  Nicholas G. Hall,et al.  Maximizing the value of a space mission , 1994 .

[3]  Al Globus,et al.  A Comparison of Techniques for Scheduling Earth Observing Satellites , 2004, AAAI.

[4]  Witold Pedrycz,et al.  An adaptive Simulated Annealing-based satellite observation scheduling method combined with a dynamic task clustering strategy , 2014, ArXiv.

[5]  Peter Norvig,et al.  Planning and Scheduling for Fleets of Earth Observing Satellites , 2001 .

[6]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[7]  Francine N. Nelson.  Scheduling Optimization for Imagery Satellite Constellations Using Column Generation , 2012 .

[8]  Zhen Yang,et al.  Online scheduling of image satellites based on neural networks and deep reinforcement learning , 2019, Chinese Journal of Aeronautics.

[9]  Gilbert Laporte,et al.  A heuristic for the multi-satellite, multi-orbit and multi-user management of Earth observation satellites , 2007, Eur. J. Oper. Res..

[10]  Yuval Tassa,et al.  Emergence of Locomotion Behaviours in Rich Environments , 2017, ArXiv.

[11]  Chen Hao.  An Algorithm of Cooperative Multiple Satellites Mission Planning Based on Multi-agent Reinforcement Learning , 2011 .

[12]  Thomas Schiex,et al.  Russian Doll Search for Solving Constraint Optimization Problems , 1996, AAAI/IAAI, Vol. 1.

[13]  Daniel Vanderpooten,et al.  Enumeration and interactive selection of efficient paths in a multiple criteria graph for scheduling an earth observing satellite , 2002, Eur. J. Oper. Res..

[14]  Jin-Kao Hao,et al.  A “Logic-Constrained” Knapsack Formulation and a Tabu Algorithm for the Daily Photograph Scheduling of an Earth Observation Satellite , 2001, Comput. Optim. Appl..

[15]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[16]  Al Globus,et al.  Scheduling Earth Observing Fleets Using Evolutionary Algorithms: Problem Description and Approach , 2002 .

[17]  Yingwu Chen,et al.  Multi satellites scheduling algorithm based on task merging mechanism , 2014, Appl. Math. Comput..