Spacecraft attitude control using path integral method via riemann manifold hamiltonian monte carlo

The separation principle for control and estimation is the traditional method for stochastic optimal control of linear systems. However, such an approach does not hold true as systems become increasingly nonlinear. This paper presents a new method for control through estimation based on the Path Integral (PI) formulation and the Riemann Manifold Hamiltonian Monte Carlo (RMHMC) method. The approach uses the path integral method to formulate the control problem as an estimation problem and adds the estimation of the model parameters to the problem. Then, a solution is found by solving the estimation problem by Riemann Manifold Hamiltonian Monte Carlo (RMHMC) sampling which includes a solution for the control parameters. By solving the nonlinear control and estimation problem using the path integral method via RMHMC, no open algorithmic tuning parameters other than exploration noise are required and the control solution has numerically robust performance to high dimensionality in the system control input. The methodology is specifically applied to the spacecraft attitude control problem, though it should be noted that such an approach is generalizable. Simulation results are presented which demonstrate good performance when utilizing the PI via RMHMC method.

[1]  Matthias Morzfeld,et al.  Path Integral Formulation of Stochastic Optimal Control with Generalized Costs , 2014, 1406.7869.

[2]  W. MacKunis,et al.  Adaptive satellite attitude control in the presence of inertia and CMG gimbal friction uncertainties , 2008 .

[3]  Hilbert J. Kappen,et al.  Graphical Model Inference in Optimal Control of Stochastic Multi-Agent Systems , 2008, J. Artif. Intell. Res..

[4]  B. Øksendal.  Stochastic Differential Equations , 1985 .

[5]  F. Y. Hadaegh,et al.  A model predictive control-based approach for spacecraft formation keeping and attitude control , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[6]  Stefano Di Cairano,et al.  Robust dual control MPC with guaranteed constraint satisfaction , 2014, 53rd IEEE Conference on Decision and Control.

[7]  Stefan Schaal,et al.  A Generalized Path Integral Control Approach to Reinforcement Learning , 2010, J. Mach. Learn. Res..

[8]  Radford M. Neal.  MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.

[9]  Evangelos A. Theodorou,et al.  Model Predictive Path Integral Control: From Theory to Parallel Computation , 2017 .

[10]  John Junkins,et al.  Optimal Estimation of Dynamic Systems, Second Edition , 2011 .

[11]  Vladimír Havlena,et al.  Multiple-Step Active Control with Dual Properties , 2011 .

[12]  J. Yong.  Relations among ODEs, PDEs, FSDEs, BSDEs, and FBSDEs , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[13]  Björn Wittenmark,et al.  Adaptive Dual Control Methods: An Overview , 1995 .

[14]  H. Kappen.  An introduction to stochastic control theory, path integrals and reinforcement learning , 2007 .

[15]  Ashish Tewari,et al.  Optimal nonlinear tracking of spacecraft attitude maneuvers , 2004, IEEE Transactions on Control Systems Technology.

[16]  E. J. Lefferts,et al.  Kalman Filtering for Spacecraft Attitude Estimation , 1982 .

[17]  Son-Goo Kim,et al.  Kalman filtering for relative spacecraft attitude and position estimation , 2005 .

[18]  Rush D. Robinett,et al.  NEW PENALTY FUNCTIONS AND OPTIMAL CONTROL FORMULATION FOR SPACECRAFT ATTITUDE CONTROL PROBLEMS , 1997 .

[19]  Hilbert J. Kappen,et al.  Stochastic optimal control of state constrained systems , 2011, Int. J. Control.

[20]  Vladimír Havlena,et al.  MPC‐based approximate dual controller by information matrix maximization , 2013 .

[21]  J. Junkins,et al.  Analytical Mechanics of Space Systems , 2003 .