Simple Sensitivity Analysis for Orion GNC

The Orion GNC entry team is analyzing the performance of Orion flight software in part by running Monte Carlo simulations of Orion spacecraft flights. The simulated performance is checked for conformance with flight requirements, expressed as performance constraints. Flight requirements include guidance (e.g., touchdown distance from target) and control (e.g., control saturation) as well as performance (e.g., heat load constraints). The Monte Carlo simulations disperse hundreds of simulation input variables, for everything from mass properties to date of launch. We describe in this paper a sensitivity analysis tool ("Critical Factors Tool" or CFT) developed to find the input variables or pairs of variables which by themselves significantly influence satisfaction of requirements or significantly affect key performance metrics (e.g., touchdown distance from target). Knowing these factors can inform robustness analysis and where engineering resources are most needed, and could even affect operations. The contributions of this paper include the introduction of novel sensitivity measures, such as estimating success probability, and a technique for determining whether pairs of factors are interacting dependently or independently. The tool found that input variables such as moments, mass, thrust dispersions, and date of launch are significant factors for success of various requirements. Examples are shown in this paper as well as a summary and physics discussion of EFT-1 driving factors that the tool found.

[1]  Shigeru Obayashi,et al.  Data Mining for Aerodynamic Design Space , 2005, J. Aerosp. Comput. Inf. Commun..

[2]  Laurent El Ghaoui,et al.  Safe Feature Elimination in Sparse Supervised Learning , 2010, ArXiv.

[3]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[4]  DAVID G. KENDALL,et al.  Introduction to Mathematical Statistics , 1947, Nature.

[5]  Tim Menzies,et al.  Parametric Analysis of a Hover Test Vehicle using Advanced Test Generation and Data Analysis , 2009 .

[6]  Trevor Hastie,et al.  Regularization Paths for Generalized Linear Models via Coordinate Descent. , 2010, Journal of statistical software.

[7]  Stefano Tarantola,et al.  Sensitivity Analysis in Practice , 2002 .

[8]  Oliver Kramer,et al.  Unsupervised K-Nearest Neighbor Regression , 2011, ArXiv.

[9]  John E. Hurtado,et al.  Tool for Rapid Analysis of Monte Carlo Simulations , 2013 .

[10]  Andrea Saltelli et al. Sensitivity analysis in practice , 2013 .

[11]  Homayoon Dezfuli,et al.  Bayesian Inference for NASA Probabilistic Risk and Reliability Analysis , 2009 .

[12]  Tim Menzies,et al.  PARAMETRIC ANALYSIS OF ANTARES RE-ENTRY GUIDANCE ALGORITHMS USING ADVANCED TEST GENERATION AND DATA ANALYSIS , 2008 .

[13]  Joel R. Bock,et al.  A New Method to Estimate Ligand-Receptor Energetics* , 2002, Molecular & Cellular Proteomics.

[14]  Carolina Restrepo,et al.  An Analysis Tool for Flight Dynamics Monte Carlo Simulations , 2011 .

[15]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[16]  A. O'Hagan,et al.  Probabilistic sensitivity analysis of complex models: a Bayesian approach , 2004 .

[17]  Carolina Restrepo,et al.  Pattern Recognition for a Flight Dynamics Monte Carlo Simulation , 2011 .

[18]  Tim Menzies,et al.  Automatically finding the control variables for complex system behavior , 2010, Automated Software Engineering.

[19]  Brian R. Hoelscher Orion Entry, Descent, and Landing Simulation , 2007 .

[20]  Santanu Das,et al.  Near Real-Time Optimal Prediction of Adverse Events in Aviation Data , 2010 .

[21]  Dirk P. Kroese,et al.  Kernel density estimation via diffusion , 2010, 1011.2602.

[22]  Mark Perry,et al.  Artificial Intelligence, Robotics and Automation in Space , 1999 .

[23]  Oliver Kramer,et al.  Dimensionality Reduction by Unsupervised K-Nearest Neighbor Regression , 2011, 2011 10th International Conference on Machine Learning and Applications and Workshops.

[24]  Sudhir Raman Bayesian grouped variable selection , 2012 .

[25]  Tim Menzies,et al.  Just enough learning (of association rules): the TAR2 “Treatment” learner , 2006, Artificial Intelligence Review.

[26]  Anthony C. Barrett A Parametric Testing Environment for Finding the Operational Envelopes of Simulated Guidance Algorithms , 2011 .

[27]  Saltelli Andrea,et al.  Global Sensitivity Analysis: The Primer , 2008 .