Expert judgments in the cost-effectiveness analysis of resource allocations: a case study in military planning

We consider resource allocation problems in which agents are assigned to tasks with the aim of (1) minimizing the costs of assigning the agents and (2) maximizing the overall value resulting from the completion of tasks. Often, such assignment problems are challenging, because it may not be known to what extent the agents can complete tasks or what the value of either full or partial task completion is. Furthermore, it may be difficult to determine how important the tasks are relative to each other. In this paper, we therefore develop an optimization framework that helps determine for a range of levels of resource expenditure (1) which combinations of agents are cost-effective and (2) to which tasks these agents should be assigned. The parameters for the optimization problem can be derived, for instance, by eliciting evaluation judgments from experts. We also provide tools for analyzing which combinations of agents outperform others in view of the judgments of all experts, and which ones are cost-ineffective based on the judgments of some or all experts. A computational algorithm is presented, and the framework is illustrated by reporting a real application in military planning.

[1]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[2]  Mark A. Burgman,et al.  What Is Expert Knowledge, How Is Such Knowledge Gathered, and How Do We Use It to Address Questions in Landscape Ecology? , 2012 .

[3]  Xavier Gandibleux,et al.  A survey and annotated bibliography of multiobjective combinatorial optimization , 2000, OR Spectr..

[4]  Carl-Axel S. Staël von Holstein,et al.  Exceptional Paper---Probability Encoding in Decision Analysis , 1975 .

[5]  Robin L. Dillon,et al.  The Respective Roles of Risk and Decision Analyses in Decision Support , 2006, Decis. Anal..

[6]  W. Knechel,et al.  Resource Allocation Decisions in Audit Engagements , 1997 .

[7]  P. Gärdenfors.  Assignment Problem Based on Ordinal Preferences , 1973 .

[8]  H. Kuhn.  The Hungarian method for the assignment problem , 1955 .

[9]  Christian Genest,et al.  Combining Probability Distributions: A Critique and an Annotated Bibliography , 1986 .

[10]  Subhash C. Narula,et al.  A multicriteria assignment problem , 2002 .

[11]  Mauro Dell'Amico,et al.  8. Quadratic Assignment Problems: Algorithms , 2009 .

[12]  John E. Beasley,et al.  A genetic algorithm for the generalised assignment problem , 1997, Comput. Oper. Res..

[13]  P. Schoemaker.  Scenario Planning: A Tool for Strategic Thinking , 1995 .

[14]  A. S. Manne.  A Target-Assignment Problem , 1958 .

[15]  Hamdy A. Taha,et al.  Operations research: an introduction / Hamdy A. Taha , 1982 .

[16]  D. N. Kleinmuntz.  Advances in Decision Analysis: Resource Allocation Decisions , 2007 .

[17]  Arvind Kumar,et al.  Exact and Heuristic Algorithms for the Weapon-Target Assignment Problem , 2007, Oper. Res..

[18]  Peter A. Morris,et al.  Decision Analysis Expert Use , 1974 .

[19]  David W. Pentico,et al.  Assignment problems: A golden anniversary survey , 2007, Eur. J. Oper. Res..

[20]  Mauro Dell'Amico,et al.  Assignment Problems , 1998, IFIP Congress: Fundamentals - Foundations of Computer Science.

[21]  Kyung S. Park,et al.  Tools for interactive multiattribute decisionmaking with incompletely identified information , 1997 .

[22]  Ahti Salo,et al.  Robust portfolio modeling with incomplete cost information and project interdependencies , 2008, Eur. J. Oper. Res..

[23]  H. Leonard.  Elicitation of Honest Preferences for the Assignment of Individuals to Positions , 1983, Journal of Political Economy.

[24]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[25]  Ahti Salo,et al.  Cost-efficiency analysis of weapon system portfolios , 2012, Eur. J. Oper. Res..

[26]  Daniel E. O'Leary,et al.  Determining Differences in Expert Judgment: Implications for Knowledge Acquisition and Validation* , 1993 .

[27]  Jh van Vuuren,et al.  Threat evaluation and weapon assignment decision support: A review of the state of the art , 2007 .

[28]  Ahti Salo,et al.  Rank inclusion in criteria hierarchies , 2005, Eur. J. Oper. Res..

[29]  E. Keeler,et al.  Editorial : Uses of cost-benefit analysis , 1987 .

[30]  William J. Tarantino,et al.  Use of Decision Analysis in the Army Base Realignment and Closure (BRAC) 2005 Military Value Analysis , 2006, Decis. Anal..

[31]  Ralph L. Keeney,et al.  Eliciting probabilities from experts in complex technical problems , 1991 .

[32]  H. Levy.  Stochastic dominance and expected utility: survey and analysis , 1992 .

[33]  Rakesh K. Sarin,et al.  Measurable Multiattribute Value Functions , 1979, Oper. Res..

[34]  Jeffrey M. Keisler,et al.  Portfolio decision analysis : improved methods for resource allocation , 2011 .

[35]  B. Dowd,et al.  Corporate benefit policies and health insurance costs. , 1984, Journal of health economics.

[36]  Ahti Salo,et al.  Preference programming for robust portfolio modeling and project selection , 2007, Eur. J. Oper. Res..

[37]  Christian Stummer,et al.  Interactive R&D portfolio analysis with project interdependencies and time profiles of multiple objectives , 2003, IEEE Trans. Engineering Management.

[38]  Ralph L. Keeney,et al.  On the uses of expert judgment on complex technical problems , 1989 .

[39]  F. Fernández,et al.  Multi-criteria analysis with partial information about the weighting coefficients , 1995 .

[40]  Gilberto Montibeller,et al.  Combining scenario planning and multi-criteria decision analysis in practice , 2006 .

[41]  Gregory S. Parnell,et al.  Blue Horizons Study Assesses Future Capabilities and Technologies for the United States Air Force , 2011, Interfaces.

[42]  T. Stewart.  Robustness of Additive Value Function Methods in MCDM , 1996 .

[43]  Ahmed Bufardi,et al.  On the Efficiency of Feasible Solutions of a Multicriteria Assignment Problem , 2008 .

[44]  A A Stinnett,et al.  Mathematical programming for the efficient allocation of health care resources. , 1996, Journal of health economics.

[45]  Simon French,et al.  Aggregating expert judgement , 2011 .

[46]  D B Evans,et al.  Development of WHO guidelines on generalized cost-effectiveness analysis. , 2000, Health economics.