Examples of Mental Mistakes Made by Systems Engineers While Creating Tradeoff Studies

Problem statement: Humans often make poor decisions. To help them make better decisions, engineers are taught to create tradeoff studies. However, these engineers are usually unaware of mental mistakes that they make while creating their tradeoff studies. We need to increase awareness of a dozen specific mental mistakes that engineers commonly make while creating tradeoff studies. Aims of the research: To prove that engineers actually do make mental mistakes while creating tradeoff studies. To identify which mental mistakes can be detected in tradeoff study documentation. Methodology: Over the past two decades, teams of students and practicing engineers in Bahill’s Systems Engineering courses wrote the system design documents for an assigned system. On average, each of these document sets took 100 man-hours to create and comprised 75 pages. We used 110 of these projects, two dozen government contractor tradeoff studies and three publicly accessible tradeoff studies. We scoured these document sets looking for examples of 28 specific mental mistakes that might affect a tradeoff study. We found instances of a dozen of these mental mistakes. Results: Often evidence of some of these mistakes cannot be found in the final documentation. To find evidence for such mistakes, the experimenters would have had to be a part of the data collection and decision making process. That is why, in this paper, we present only 12 of the original 28 mental mistakes. We found hundreds of examples of such mistakes. We provide suggestions to help people avoid making these mental mistakes while doing tradeoff studies. Conclusions: This paper shows evidence of a dozen common mental mistakes that are continually being repeated by engineers while creating tradeoff studies. When engineers are taught about these mistakes, they can minimize their occurrence in the future.

[1]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[2]  D. Kahneman MAPS OF BOUNDED RATIONALITY: A PERSPECTIVE ON INTUITIVE JUDGMENT AND CHOICE , 2003 .

[3]  A. Tversky,et al.  The Disjunction Effect in Choice under Uncertainty , 1992 .

[4]  A. Terry Bahill,et al.  Predicting a Baseball’s Path , 2005 .

[5]  Stephen M. Robinson,et al.  Reducing Perceptual and Cognitive Challenges in Making Decisions with Models , 2008 .

[6]  Carmen Tanner,et al.  Taboos and conflicts in decision making: Sacred values, decision difficulty, and emotions , 2008, Judgment and Decision Making.

[7]  Eric D. Smith,et al.  Sensitivity Analysis, a Powerful System Validation Technique , 2007 .

[8]  A. Terry Bahill,et al.  Paradoxical pop-ups: Why are they difficult to catch? , 2008, 0803.4357.

[9]  B. R. Forer,et al.  The fallacy of personal validation; a classroom demonstration of gullibility. , 2011, Journal of abnormal psychology.

[10]  A. Terry Bahill,et al.  Engineering Modeling and Design , 1992 .

[11]  A. Terry Bahill,et al.  Quantitative methods for tradeoff analyses , 2001 .

[12]  A. Bahill,et al.  Why can't batters keep their eyes on the ball? , 1984 .

[13]  Lawrence D. Pohlmann Strategics Consulting Customer-Centered Products – Creating Successful Products through Smart Requirements Management , 2015 .

[14]  A. T. Bahill,et al.  Discovering system requirements , 1996 .

[15]  Liming Zhu,et al.  Tradeoff and Sensitivity Analysis in Software Architecture Evaluation Using Analytic Hierarchy Process , 2005, Software Quality Journal.

[16]  Andrew P. Sage,et al.  Behavioral and Organizational Considerations in the Design of Information Systems and Processes for Planning and Decision Support , 1981, IEEE Transactions on Systems, Man, and Cybernetics.

[17]  David G. Baldwin,et al.  Effects of Altitude and Atmospheric Conditions on the Flight of a Baseball , 2009 .

[18]  M. Abdellaoui Parameter-Free Elicitation of Utility and Probability Weighting Functions , 2000 .

[19]  A. Terry Bahill,et al.  Describing baseball pitch movement with right-hand rules , 2007, Comput. Biol. Medicine.

[20]  Eric D. Smith,et al.  Cognitive Biases Affect the Acceptance of Tradeoff Studies , 2008 .

[21]  Eric D. Smith,et al.  An Industry Standard Risk Analysis Technique , 2009 .

[22]  A. Wayne Wymore,et al.  Model-based systems engineering , 1993 .

[23]  A. Terry Bahill,et al.  16 of Baseball Pitching and Batting , 2008 .

[24]  Mary Beth Chrissis,et al.  CMMI: Guidelines for Process Integration and Product Improvement , 2003 .

[25]  A. Terry Bahill,et al.  The hybrid process that combines traditional requirements and use cases , 2004, Syst. Eng..

[26]  A. Terry Bahill,et al.  Sensitivity analyses of continuous and discrete systems in the time and frequency domains , 1993, IEEE Trans. Syst. Man Cybern..

[27]  J. L. Pinto,et al.  A Parameter-Free Elicitation of the Probability Weighting Function in Medical Decision Analysis , 2000 .

[28]  L Stark,et al.  Oblique saccadic eye movements. Independence of horizontal and vertical channels. , 1977, Archives of ophthalmology.

[29]  D. Kahneman A perspective on judgment and choice: mapping bounded rationality. , 2003, The American psychologist.

[30]  Reshma Jagsi,et al.  Coordinating Cancer Care: Patient and Practice Management Processes Among Surgeons Who Treat Breast Cancer , 2010, Medical care.

[31]  A. Terry Bahill,et al.  Ameliorating mental mistakes in tradeoff studies , 2007 .

[32]  Nora Miller,et al.  Inevitable Illusions: How Mistakes of Reason Rule Our Minds , 2001 .

[33]  L. Stark,et al.  Neurological control of horizontal and vertical components of oblique saccadic eye movements , 1975 .

[34]  S. French Value‐focused thinking: A path to creative decision making, Keeney, R. L., Cambridge, MA: Harvard University Press, 1992, ISBN 0‐674‐93197‐1 , 1993 .

[35]  A. Terry Bahill,et al.  Risk analysis of a Pinewood derby : A case study , 2000 .

[36]  L Stark,et al.  Parametric sensitivity analysis of a homeomorphic model for saccadic and vergence eye movements. , 1976, Computer programs in biomedicine.