Do Sensitivity Analyses Really Capture Problem Sensitivity? An Empirical Analysis Based on Information Value

The most common methods of sensitivity analysis (SA) in decision-analytic modeling are based either on proximity in parameter-space to decision thresholds or on the range of payoffs that accompany parameter variation. As an alternative, we propose the use of the expected value of perfect information (EVPI) as a sensitivity measure and argue from first principles that it is the proper measure of decision sensitivity. EVPI has significant advantages over conventional SA, especially in the multiparametric case, where graphical SA breaks down. In realistically sized problems, simple oneand two-way SAs may not fully capture parameter interactions, raising the disturbing possibility that many published decision analyses might be overconfident in their policy recommendations. To investigate the extent of this potential problem, we re-examined 25 decision analyses drawn from the published literature and calculated EVPI values for parameters on which sensitivity analyses had been performed, as well as the entire set of problem parameters. While we expected EVPI values to indicate greater problem sensitivity than conventional SA due to revealed parameter interaction, we in fact found the opposite: compared to EVPI, the oneand twoparameter SAs accompanying these problems dramatically overestimated problem sensitivity to input parameters. This phenomenon can be explained by invoking the flat maxima principle enunciated by von Winterfeldt and Edwards.