Ameliorating mental mistakes in tradeoff studies

Tradeoff studies are broadly recognized and mandated as the method for simultaneously considering multiple alternatives with many criteria, and as such are recommended in the Capability Maturity Model Integration (CMMI) Decision Analysis and Resolution (DAR) process. Tradeoff studies, which involve human numerical judgment, calibration, and data updating, are often approached with under confidence by analysts and are often distrusted by decision makers. The decision-making fields of Judgment and Decision Making, Cognitive Science and Experimental Economics have built up a large body of research on human biases and errors in considering numerical and criteria-based choices. Relationships between experiments in these fields and the elements of tradeoff studies show that tradeoff studies are susceptible to human mental mistakes: This paper indicates ways to eliminate the presence, or ameliorate the effects of mental mistakes on tradeoff studies. © 2007 Wiley Periodicals, Inc. Syst Eng 10: 222–240, 2007 This work was supported by the Air Force Office of Scientific Research under AFOSR/MURI F49620-03-1-0377.

[1]  Dennis M. Buede,et al.  The Engineering Design of Systems: Models and Methods , 1999 .

[2]  B. R. Forer,et al.  The fallacy of personal validation; a classroom demonstration of gullibility. , 2011, Journal of abnormal psychology.

[3]  Kristin A. Farry,et al.  Customer Centered Products: Creating Successful Products Through Smart Requirements Management (Продукты для клиента: создание успешных продуктов с помощью грамотного управления требованиями) , 2000 .

[4]  P. Slovic,et al.  Reversals of preference between bids and choices in gambling decisions. , 1971 .

[5]  A. Tversky,et al.  Prospect Theory : An Analysis of Decision under Risk Author ( s ) : , 2007 .

[6]  Ilana Ritov,et al.  Determinants of stated willingness to pay for public goods: A study in the headline method , 1994, Journal of risk and uncertainty.

[7]  B. Fischhoff,et al.  Reasons for confidence. , 1980 .

[8]  H. Simon,et al.  A Behavioral Model of Rational Choice , 1955 .

[9]  A. Terry Bahill,et al.  Sensitivity analyses of continuous and discrete systems in the time and frequency domains , 1993, IEEE Trans. Syst. Man Cybern..

[10]  B. Fischhoff,et al.  On the Psychology of Experimental Surprises. , 1977 .

[11]  David L. Mothersbaugh,et al.  Consumer Knowledge Assessment , 1994 .

[12]  D. Schacter.  Feeling of knowing in episodic memory. , 1983 .

[13]  H. Simon,et al.  Perception in chess , 1973 .

[14]  Holger R. Maier,et al.  Sensitivity and Uncertainty , 2008 .

[15]  Terry Bahill,et al.  The hybrid process that combines traditional requirements and use cases , 2004 .

[16]  A. Terry Bahill,et al.  A Tutorial on Quality Function Deployment , 1993 .

[17]  A. Terry Bahill,et al.  Risk analysis of a Pinewood derby : A case study , 2000 .

[18]  Lee Roy Beach,et al.  The Psychology of Decision Making: People in Organizations , 1997 .

[19]  H. Arkes,et al.  Impediments to accurate clinical judgment and possible ways to minimize their impact. , 1981, Journal of consulting and clinical psychology.

[20]  Massimo Piattelli-Palmarini,et al.  Inevitable Illusions: How Mistakes of Reason Rule Our Minds , 1994 .

[21]  Leonard E. Miller,et al.  NASA systems engineering handbook , 1995 .

[22]  D. Cacuci,et al.  SENSITIVITY and UNCERTAINTY ANALYSIS , 2003 .

[23]  G. Ainslie.  Breakdown of will , 2001 .

[24]  R. Nickerson.  Confirmation Bias: A Ubiquitous Phenomenon in Many Guises , 1998 .

[25]  H. Simon,et al.  Models of Man. , 1957 .

[26]  Yacov Y. Haimes,et al.  Risk modeling, assessment, and management , 1998 .

[27]  A. Tversky,et al.  The framing of decisions and the psychology of choice. , 1981, Science.

[28]  Eric D. Smith,et al.  Sensitivity Analysis, a Powerful System Validation Technique , 2007 .

[29]  Thomas L. Saaty,et al.  Multicriteria Decision Making: The Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation , 1990 .

[30]  G. Gigerenzer.  How to Make Cognitive Illusions Disappear: Beyond “Heuristics and Biases” , 1991 .

[31]  J. Dombi,et al.  A method for determining the weights of criteria: the centralized weights , 1986 .

[32]  H. Simon,et al.  Models Of Man : Social And Rational , 1957 .

[33]  N. A. Rahman,et al.  How to solve problems : elements of a theory of problems and problem solving , 1975 .

[34]  Ward Edwards,et al.  How to Use Multiattribute Utility Measurement for Social Decisionmaking , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[35]  A. Wayne Wymore,et al.  Model-based systems engineering , 1993 .

[36]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[37]  Martin Weber,et al.  Behavioral influences on weight judgments in multiattribute decision making , 1993 .

[38]  Ralph L. Keeney,et al.  Value-Focused Thinking: A Path to Creative Decisionmaking , 1992 .

[39]  D. Kumaran,et al.  Frames, Biases, and Rational Decision-Making in the Human Brain , 2006, Science.

[40]  A. T. Bahill,et al.  Discovering system requirements , 1996 .

[41]  A. Tversky,et al.  Contingent weighting in judgment and choice , 1988 .

[42]  Y. Haimes.  Risk Modeling, Assessment, and Management: Haimes/Risk Modeling, Assessment 2e , 2005 .

[43]  R. L. Keeney,et al.  Decisions with Multiple Objectives: Preferences and Value Trade-Offs , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[44]  H. Raiffa,et al.  The hidden traps in decision making. , 1998, Harvard business review.

[45]  David G. Ullman,et al.  Trade Studies with Uncertain Information , 2006 .

[46]  Andrew P. Sage,et al.  Concise encyclopedia of information processing in systems &organizations , 1990 .

[47]  Carolyn Y. Woo,et al.  Entrepreneurs' perceived chances for success , 1988 .

[48]  Ralph L. Keeney,et al.  Common Mistakes in Making Value Trade-Offs , 2002, Oper. Res..

[49]  Terry Bahill,et al.  The Zachman Framework Populated with Baseball Models , 2006 .

[50]  D. Redelmeier,et al.  Medical decision making in situations that offer multiple alternatives. , 1995, JAMA.

[51]  Christopher K. Hsee,et al.  Preference Reversals between Joint and Separate Evaluations of Options: A Review and Theoretical Analysis , 1999 .

[52]  A. Tversky,et al.  The weighing of evidence and the determinants of confidence , 1992, Cognitive Psychology.

[53]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[54]  Henry Petroski.  Framing Hypotheses: A Cautionary Tale , 2003 .

[55]  Donald E. Campbell,et al.  Realization of Choice Functions , 1978 .

[56]  A. Terry Bahill,et al.  Engineering Modeling and Design , 1992 .

[57]  D. Kahneman,et al.  Representativeness revisited: Attribute substitution in intuitive judgment. , 2002 .

[58]  A. Tversky,et al.  Advances in prospect theory: Cumulative representation of uncertainty , 1992 .

[59]  Eric D. Smith.  Tradeoff Studies and Cognitive Biases , 2006 .

[60]  D. Kahneman.  A perspective on judgment and choice: mapping bounded rationality. , 2003, The American psychologist.

[61]  A. Tversky,et al.  The Disjunction Effect in Choice under Uncertainty , 1992 .

[62]  Norbert Schwarz,et al.  Priming and communication: Social determinants of information use in judgments of life satisfaction , 1988 .

[63]  Stefano Tarantola,et al.  Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models , 2004 .

[64]  Gary Klein,et al.  Sources of Power: How People Make Decisions , 2017 .

[65]  A. Terry Bahill,et al.  Quantitative methods for tradeoff analyses , 2001 .

[66]  A. Terry Bahill,et al.  The systems engineering started in the middle process: A consensus of systems engineers and project managers , 2001 .