Propagating and Mitigating Uncertainty in the Design of Complex Multidisciplinary Systems

As humanity has developed increasingly ingenious and complicated systems, it has not been able to accurately predict the performance, development time, reliability, or cost of such systems. This inability to accurately predict parameters of interest in the design of complex multidisciplinary systems such as automobiles, aircraft, or spacecraft is due in great part to uncertainty. Uncertainty in complex multidisciplinary system design is currently mitigated through the use of heuristic margins. The use of these heuristic margins can result in a system being overdesigned during development or failing during operation. This thesis proposes a formal method to propagate and mitigate uncertainty in the design of complex multidisciplinary systems. Specifically, applying the proposed method produces a rigorous foundation for determining design margins. The method comprises five distinct steps: identifying tradable parameters; generating analysis models; classifying and addressing uncertainties; quantifying interaction uncertainty; and determining margins, analyzing the design, and trading parameters. The five steps of the proposed method are defined in detail. Margins are now a function of risk tolerance and are measured relative to mean expected system performance, not variations in design parameters measured relative to heuristic values. As an example, the proposed method is applied to the preliminary design of a spacecraft attitude determination and control system. In particular, the design of the attitude control system on the Mars Exploration Rover spacecraft cruise stage is used. Use of the proposed method for the example presented yields significant differences between the calculated design margins and the values assumed by the Mars Exploration Rover project. In addition to providing a formal and rigorous method for determining design margins, this thesis provides three other principal contributions. The first is an uncertainty taxonomy for use in the design of complex multidisciplinary systems with detailed definitions for each uncertainty type. The second is the modification of two simulation techniques, the mean value method and subset simulation, that can significantly reduce the computational burden in applying the proposed method. The third is a set of diverse application examples and various simulation techniques that demonstrate the generality and benefit of the proposed method.

[1]  Ronald A. Howard,et al.  Decision analysis: Perspectives on inference, decision, and experimentation , 1970 .

[2]  I. Miller Probability, Random Variables, and Stochastic Processes , 1966 .

[3]  Durward K. Sobek,et al.  The Second Toyota Paradox: How Delaying Decisions Can Make Better Cars Faster , 1995 .

[4]  William Sauway Law,et al.  Evaluating imprecision in engineering design , 1996 .

[5]  Derek J. Pike,et al.  Empirical Model‐building and Response Surfaces. , 1988 .

[6]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[7]  Erik K. Antonsson,et al.  Tuning parameters in engineering design , 1993 .

[8]  Ilan Kroo,et al.  Collaborative Approach to Launch Vehicle Design , 1997 .

[9]  E. Saliby Descriptive Sampling: An Improvement Over Latin Hypercube Sampling , 1997, Winter Simulation Conference Proceedings,.

[10]  Marcel J. Sidi,et al.  Spacecraft Dynamics and Control: Contents , 1997 .

[11]  M. Deaton,et al.  Response Surfaces: Designs and Analyses , 1989 .

[12]  J. Beck,et al.  Estimation of Small Failure Probabilities in High Dimensions by Subset Simulation , 2001 .

[13]  V T Covello,et al.  Decision analysis and risk management decision making: issues and methods. , 1987, Risk analysis : an official publication of the Society for Risk Analysis.

[14]  Tyson R. Browning,et al.  Modeling and analyzing cost, schedule, and performance in complex system product development , 1998 .

[15]  Chris Edwards Government Schemes Cost More Than Promised , 2003 .

[16]  Gerhard Pahl,et al.  Process of Planning and Designing , 1996 .

[17]  R. Roncoli,et al.  Mission design overview for the Mars Exploration Rover Mission , 2002 .

[18]  P. Wirsching,et al.  Advanced Reliability Methods for Structural Evaluation , 1987 .

[19]  Bong Wie,et al.  Space Vehicle Dynamics and Control , 1998 .

[20]  Richard J. Beckman,et al.  A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code , 2000, Technometrics.

[21]  Todd Mosher,et al.  Conceptual Spacecraft Design Using a Genetic Algorithm Trade Selection Process , 1999 .

[22]  K. Denbigh,et al.  Thermodynamic Functions of Gases. : Edited by F. Din, Butterworths Scientific Publications, 1956. Vol I 175 p. + vi, Vol. II, 201 p. + vi. 63s. each volume. , 1956 .

[23]  Wei Chen,et al.  Methodology for Managing the Effect of Uncertainty in Simulation-Based Design , 2000 .

[24]  J. Keynes The General Theory of Employment , 1937 .

[25]  Daniel DeLaurentis A probabilistic approach to aircraft design emphasizing stability and control uncertainties , 1998 .

[26]  M. Elisabeth Paté-Cornell,et al.  Bayesian analysis of launch vehicle reliability , 2003 .

[27]  Carl-Axel S. Staël von Holstein,et al.  Exceptional Paper---Probability Encoding in Decision Analysis , 1975 .

[28]  S. R. Jammalamadaka,et al.  Against the Gods: The Remarkable Story of Risk , 1999 .

[29]  George K. Maglaras,et al.  Experimental comparison of probabilistic methods and fuzzy sets for designing under uncertainty , 1995 .

[30]  Colin Camerer,et al.  Recent developments in modeling preferences: Uncertainty and ambiguity , 1992 .

[31]  James R. Wertz,et al.  Space Mission Analysis and Design , 1992 .

[32]  James R. Wertz,et al.  Spacecraft attitude determination and control , 1978 .

[33]  H. Saunders,et al.  Probabilistic Engineering Design—Principles and Applications , 1987 .

[34]  R. Ylmaz Arguden,et al.  Understanding the Outcomes of Mega-Projects , 1988 .

[35]  Annalisa L. Weigel,et al.  Bringing policy into space systems conceptual design : qualitative and quantitative methods , 2002 .

[36]  Daniel P. Thunnissen,et al.  Method for Determining Margins in Conceptual Design , 2004 .

[37]  Daniel P. Thunnissen,et al.  Assessing model uncertainty in the conceptual design of a monopropellant propulsion system , 2003 .

[38]  M. David Burghardt Introduction to Engineering Design and Problem Solving , 1998 .

[39]  Ian Hacking,et al.  The Emergence of Probability. A Philosophical Study of Early Ideas about Probability, Induction and Statistical Inference , 1979 .

[40]  Howard Raiffa,et al.  Decision analysis: introductory lectures on choices under uncertainty. 1968. , 1969, M.D.Computing.

[41]  D. P. Thunnissen,et al.  Icemaker/sup TM/: an excel-based environment for collaborative design , 2003, 2003 IEEE Aerospace Conference Proceedings (Cat. No.03TH8652).

[42]  R. T. Cox The Algebra of Probable Inference , 1962 .

[43]  Steven Vajda,et al.  Games and Decisions. By R. Duncan Luce and Howard Raiffa. Pp. xi, 509. 70s. 1957. (J Wiley & Sons) , 1959, The Mathematical Gazette.

[44]  Baruch Fischhoff,et al.  Understanding long-term environmental risks , 1990 .

[45]  Karl Borch THE ECONOMICS OF UNCERTAINTY, X , 1965 .

[46]  R. Burchfield Oxford English dictionary , 1982 .

[47]  ERRORS AND UNCERTAINTIES IN PROBABILISTIC ENGINEERING ANALYSIS , 2001 .

[48]  Ralph L. Keeney,et al.  Decisions with multiple objectives: preferences and value tradeoffs , 1976 .

[49]  Christopher D. Hall,et al.  Spacecraft Dynamics and Control , 2002 .

[50]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[51]  Erik K. Antonsson,et al.  Imprecision in Engineering Design , 1995 .

[52]  Erik K. Antonsson,et al.  Design parameter selection in the presence of noise , 1994 .

[53]  Lotfi A. Zadeh,et al.  MAKING COMPUTERS THINK LIKE PEOPLE , 1984 .

[54]  Werner Rothengatter,et al.  Megaprojects and Risk: Conventional megaproject development , 2003 .

[55]  M. Tribus Rational descriptions, decisions, and designs , 1969 .

[56]  Max Henrion,et al.  Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis , 1990 .

[57]  David A. Cleaves,et al.  Assessing Uncertainty in Expert Judgments About Natural Resources , 1994 .

[58]  A. Mullin,et al.  My Brain is Open: The Mathematical Journeys of Paul Erdos.@@@The Man Who Loved Only Numbers: The Story of Paul Erdos and the Search for Mathematical Truth. , 1998 .

[59]  Y.-T. Wu,et al.  COMPUTATIONAL METHODS FOR EFFICIENT STRUCTURAL RELIABILITY AND RELIABILITY SENSITIVITY ANALYSIS , 1993 .

[60]  David Dequech,et al.  Fundamental Uncertainty and Ambiguity , 2000 .

[61]  Daniel P. Thunnissen,et al.  Uncertainty Classification for the Design and Development of Complex Systems , 2003 .

[62]  Lesley F. Wright,et al.  Information Gap Decision Theory: Decisions under Severe Uncertainty , 2004 .

[63]  Myles Alexander Walton Managing uncertainty in space systems conceptual design using portfolio theory , 2002 .

[64]  Robin L. Dillon Programmatic risk analysis , 2002 .

[65]  Wilson H. Tang,et al.  Probability concepts in engineering planning and design , 1984 .

[66]  Daniel P. Thunnissen,et al.  Balancing Cost, Risk, and Performance under Uncertainty in Preliminary Mission Design , 2004 .

[67]  H. Sapolsky The Polaris System Development: Bureaucratic and Programmatic Success in Government , 1972 .

[68]  Ralph L. Keeney,et al.  Eliciting probabilities from experts in complex technical problems , 1991 .

[69]  Harley Thronson,et al.  Contracting inside an organization: An experimental study , 2007 .

[70]  P Morris,et al.  Research at Oxford into the Preconditions of Success and Failure in Major Projects , 1986 .

[71]  Karl Aspelund,et al.  The Design Process , 2006 .

[72]  Ross Theodore McNutt Reducing DoD Product Development Time: The Role of the Schedule Development Process. , 1998 .

[73]  Kevin Otto,et al.  A formal representational theory for engineering design , 1992 .

[74]  Byron A. Ellis,et al.  Life Cycle Cost , 2007 .

[75]  M. Kaplan Modern spacecraft dynamics & control , 1976 .

[76]  Dimitri N. Mavris,et al.  Comparison of Two Probabilistic Techniques for the Assessment of Economic Uncertainty , 1997 .

[77]  W. Edwards,et al.  Decision Analysis and Behavioral Research , 1986 .

[78]  Eric Fox,et al.  The Pratt & Whitney Probabilistic Design System , 1994 .

[79]  J. Nash NON-COOPERATIVE GAMES , 1951, Classics in Game Theory.

[80]  Kenneth F. Alvin,et al.  A New Methodology for the Estimation of Total Uncertainty in Computational Simulation , 1999 .

[81]  Francis Sullivan,et al.  The Metropolis Algorithm , 2000, Computing in Science & Engineering.

[82]  Jon C. Helton,et al.  Mathematical representation of uncertainty , 2001 .

[83]  Glenn T. Tsuyuki,et al.  Margin determination in the design and development of a thermal control system , 2004 .

[84]  A. I. Khuri,et al.  Response Surfaces: Designs and Analyses: Second Edition , 1987 .

[85]  Zissimos P. Mourelatos,et al.  A Method for Multiattribute Automotive Design under Uncertainty , 2004 .

[86]  R. Cooke Elicitation of expert opinions for uncertainty and risks , 2003 .

[87]  Bruno O. Shubert,et al.  Random variables and stochastic processes , 1979 .

[88]  M. Resnik Choices: An Introduction to Decision Theory , 1990 .

[89]  H. Christopher Frey,et al.  Quantitative Analysis of Uncertainty and Variability in Environmental Policy Making , 1992 .

[90]  Dimitri N. Mavris,et al.  Uncertainty Modeling and Management in Multidisciplinary Analysis and Synthesis , 2000 .

[91]  Paul S. Fischbeck,et al.  Probabilistic risk analysis and risk-based priority scale for the tiles of the space shuttle , 1993 .

[92]  Daniel Thunnissen,et al.  Propagating and mitigating the effect of uncertainty in the conceptual design of a monopropellant propulsion system , 2003 .

[93]  Roy Radner,et al.  Equilibrium under uncertainty , 1993 .

[94]  R. L. Keeney,et al.  Decisions with Multiple Objectives: Preferences and Value Trade-Offs , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[95]  Michael P. Giannetto,et al.  The Man Who Loved Only Numbers , 2005 .

[96]  J. Neumann,et al.  Theory of games and economic behavior , 1945, 100 Years of Math Milestones.

[97]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[98]  Rajesh Rajamani,et al.  Vehicle dynamics and control , 2005 .

[99]  Gajanana C. Birur,et al.  Active Heat Rejection System on Mars Exploration Rover — Design Changes from Mars Pathfinder , 2003 .

[100]  L. Phillips A theory of requisite decision models , 1984 .

[101]  Vladimir A. Chobotov Spacecraft Attitude Dynamics and Control , 1991 .

[102]  Michael Joseph Scott,et al.  Formalizing Negotiation in Engineering Design , 1999 .

[103]  D Malakoff,et al.  Bayes Offers a 'New' Way to Make Sense of Numbers , 1999, Science.

[104]  Leonard E. Miller,et al.  NASA systems engineering handbook , 1995 .

[105]  Yakov Ben-Haim,et al.  Uncertainty, probability and information-gaps , 2004, Reliab. Eng. Syst. Saf..

[106]  Raymond J. Sedwick,et al.  Application of Multidisciplinary Design Optimization Techniques to Distributed Satellite Systems , 2000 .

[107]  Charles Seife Columbia Disaster Underscores the Risky Nature of Risk Analysis , 2003, Science.

[108]  Kristin Lee Wood A method for the representation and manipulation of uncertainties in preliminary engineering design , 1990 .

[109]  R. T. Cox,et al.  The Algebra of Probable Inference , 1962 .

[110]  Siu-Kui Au,et al.  On the Solution of First Excursion Problems by Simulation with Applications to Probabilistic Seismic Performance Assessment , 2001 .

[111]  Jay L. Devore,et al.  Probability and statistics for engineering and the sciences , 1982 .

[112]  Bryan E. Denham Against the Gods: The Remarkable Story of Risk , 1997 .

[113]  A. Tversky,et al.  Judgment under Uncertainty , 1982 .

[114]  J. Beck,et al.  First excursion probabilities for linear systems by very efficient importance sampling , 2001 .

[115]  Henry Petroski,et al.  Design Paradigms: Introduction , 1994 .

[116]  Edmund Conrow Effective Risk Management , 2003 .

[117]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.