Caveats for simulation modeling in support of decision making

Simulation modeling is a tool commonly used in support of intelligent decision making by senior managers, particularly for extremely complex problems. We use an example from the United States army recruiting command to illustrate some of the statistical pitfalls an analyst may encounter when using simulation modeling. These pitfalls include conflicting results, both due to different modeling approaches and choice of input distributions, and incorrect interpretation of the simulation experimental results. We also provide implications for analysts who encounter these situations. The analyst who uses simulation in support of senior decision-makers must understand simulation's capabilities, limitations, and statistical underpinnings. Failing to do so can result in decisions based on incorrect information. Analysts can guard against these pitfalls through careful consideration of statistics, preparation, and communication.