User acceptance of multi-criteria decision support systems: The impact of preference elicitation techniques

Previous research indicates that decision makers are often reluctant to use potentially beneficial multi-criteria decision support systems (MCDSS). Prior research has not examined the specific impact of preference elicitation techniques on user acceptance of MCDSS. The present research begins to fill this gap by examining the effect on users MCDSS evaluations of two commonly used preference elicitation techniques, absolute measurement and pairwise comparisons, while holding constant all other aspects of the MCDSS and decision making task. Experimental results (N = 153) indicate that users consider MCDSS with pairwise comparisons to be higher in decisional conflict, more effortful, less accurate, and overall less desirable to use than MCDSS with absolute measurements. Thus, any potential normative superiority of a preference elicitation technique must be balanced against its potentially adverse effects on user acceptance of the MCDSS within which it is employed. We present a research agenda for exploring the tradeoffs between objective validity and user acceptance in the design of decision analysis tools. 2004 Elsevier B.V. All rights reserved.

[1]  Dane K. Peterson,et al.  Confidence, uncertainty, and the use of information , 1988 .

[2]  Jyrki Wallenius,et al.  Comparative Evaluation of Some Interactive Approaches to Multicriterion Optimization , 1975 .

[3]  Charles Vlek,et al.  An Evaluation Study of the Effectiveness of Multi-attribute Decision Support as a Function of Problem Complexity , 1994 .

[4]  A. Tversky,et al.  Contingent weighting in judgment and choice , 1988 .

[5]  Ramon J. Aldag,et al.  Decision Making: A Psychological Analysis of Conflict , 1980 .

[6]  Kalyanmoy Deb,et al.  Multiple Criteria Decision Making, Multiattribute Utility Theory: Recent Accomplishments and What Lies Ahead , 2008, Manag. Sci..

[7]  G. W. Evans,et al.  An Overview of Techniques for Solving Multiobjective Mathematical Programs , 1984 .

[8]  Pai-Cheng Chu,et al.  The Joint Effects of Effort and Quality on Decision Strategy Choice with Computerized Decision Aids , 2000, Decis. Sci..

[9]  K. Brockhoff.  Experimental test of MCDM algorithms in a modular approach , 1985 .

[10]  G. Reeder,et al.  Inferences about Effort and Ability , 2001 .

[11]  M. Scholten.  Conflict-mediated choice , 2002 .

[12]  P. Gardner.  Scales and Statistics , 1975 .

[13]  V. Belton.  A comparison of the analytic hierarchy process and a simple multi-attribute value function , 1986 .

[14]  Robert F. DeVellis,et al.  Scale Development: Theory and Applications. , 1992 .

[15]  S. S. Stevens,et al.  Handbook of experimental psychology , 1951 .

[16]  Philippe Delquié,et al.  Optimal Conflict in Preference Assessment , 2003, Manag. Sci..

[17]  John W. Payne,et al.  Effort and Accuracy in Choice , 1985 .

[18]  Christopher K. Hsee,et al.  Preference Reversals between Joint and Separate Evaluations of Options: A Review and Theoretical Analysis , 1999 .

[19]  G. W. Fischer,et al.  Strategy compatibility, scale compatibility, and the prominence effect. , 1993 .

[20]  H. G. Daellenbach,et al.  A comparative evaluation of interactive solution methods for multiple objective decision models , 1987 .

[21]  Eric J. Johnson,et al.  The adaptive decision maker , 1993 .

[22]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[23]  Gerardine DeSanctis,et al.  Providing Decisional Guidance for Multicriteria Decision Making in Groups , 2000, Inf. Syst. Res..

[24]  M. F. Luce,et al.  Emotional Trade-Off Difficulty and Choice: , 1999 .

[25]  Timothy B. Heath,et al.  Conflict and Loss Aversion in Multiattribute Choice: The Effects of Trade-Off Size and Reference Dependence on Decision Difficulty☆ , 1996 .

[26]  Fred D. Davis,et al.  Computer-Assisted Decision Making: Performance, Beliefs, and the Illusion of Control , 1994 .

[27]  Gary P. Latham,et al.  THE FUTILITY OF UTILITY ANALYSIS REVISITED: WHEN EVEN AN EXPERT FAILS , 1997 .

[28]  Fred D. Davis,et al.  User Acceptance of Computer Technology: A Comparison of Two Theoretical Models , 1989 .

[29]  Yannis Siskos,et al.  Intelligent multicriteria decision support: Overview and perspectives , 1999, Eur. J. Oper. Res..

[30]  J. Hair.  Multivariate data analysis , 1972 .

[31]  Allen C. Amason.  Distinguishing the Effects of Functional and Dysfunctional Conflict on Strategic Decision Making: Resolving a Paradox for Top Management Teams , 1996 .

[32]  M. F. Luce,et al.  Correlation, conflict, and choice. , 1993 .

[33]  David L. Olson,et al.  Ordinal judgments in multiattribute decision analysis , 2002, Eur. J. Oper. Res..

[34]  Jeffrey E. Kottemann,et al.  Decisional Conflict and User Acceptance of Multicriteria Decision-Making Aids , 1991 .

[35]  Ram Narasimhan,et al.  An Experimental Evaluation of Articulation of Preferences in Multiple Criterion Decision‐Making (MCDM) Methods , 1988 .

[36]  S. Zionts,et al.  An Interactive Multiple Objective Linear Programming Method for a Class of Underlying Nonlinear Utility Functions , 1983 .

[37]  John Buchanan,et al.  Capturing decision maker preference: Experimental comparison of decision analysis and MCDM techniques , 1997 .

[38]  Eric J. Johnson,et al.  A componential analysis of cognitive effort in choice , 1990 .

[39]  Vincent Mousseau,et al.  A user-oriented implementation of the ELECTRE-TRI method integrating preference elicitation support , 2000, Comput. Oper. Res..

[40]  T. Saaty,et al.  The Analytic Hierarchy Process , 1985 .

[41]  Izak Benbasat,et al.  Inducing compensatory information processing through decision aids that facilitate effort reduction: an experimental assessment , 2000 .

[42]  E. Spires,et al.  Cross-Cultural Differences in Choice Behavior and Use of Decision Aids: A Comparison of Japan and the United States. , 1999, Organizational behavior and human decision processes.

[43]  Elazar J. Pedhazur,et al.  Measurement, Design, and Analysis: An Integrated Approach , 1994 .

[44]  Ralph E. Steuer,et al.  Multiple Criteria Decision Making, Multiattribute Utility Theory: The Next Ten Years , 1992 .

[45]  Fred D. Davis,et al.  A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies , 2000, Management Science.

[46]  Steven M. Shugan.  The Cost Of Thinking , 1980 .

[47]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[48]  David L. Olson,et al.  Consistency and Accuracy in Decision Aids: Experiments with Four Multiattribute Systems* , 1995 .